MEASURING THE TRANSOM ANGLE: Place a carpenter's square, with the long side (22.½") under the keel or to the outside edge of the hull, parallel with the chine. When the short side (14") comes in contact with the transom, take the measurement (see sketch) between the inside corner of the square and the boat transom. Each 1/4" is equal to 1° in transom angle.
This is from ArmstrongNauticals site.
http://www.armstrongnautical.com/Brackform.htm