In 1764, Samuel Leffers came to Beaufort from Hempstead, Queens County, New York, as schoolmaster. Over the years, Schoolmaster Leffers became a surveyor, clerk of court, merchant and an investor in real estate. In 1795, Samuel Leffers purchased the 25-acre White House property, including house, windmill and other improvements, from Elias Albertson Jr. for 350 Spanish dollars. 
     In an October 19, 1800 letter to his brother John in Long Island, New York, Samuel Leffers wrote, "My situation at present is agreeable, my new house is calculated to my fancy and pleasantly situated, we have a fine prospect of the Sea, in front have a good garden and spring of water and are about 200 yards from the eastern most boundary of Beaufort town." (Leffer's new house was built about 300 yards east of the White House, which was 100 yards west of the town boundary, the boundary described earlier as "100 yards to the eastward of the hammock that Thomas Austin formerly lived on.")