Remsenburg is an exclusive area of the Hamptons. There are no stoplights and very few commercial businesses.
As early as 1712, meadows in Speonk were leased to cattle-owners from Southampton. Most of the early settlers of Speonk came west from Southampton and Bridgehampton in the 1740s to clear the forests and build farms. During the 1880s, duck farms thrived in Speonk, but few survived past the turn of the century.
The name Speonk was inspired by a Native American word meaning high place. An 1897 Long Island Rail Road catalog listed Speonk, noting that that name "certainly sounds like the call of a frog." Some residents pressed to change the name to Remsenburg, to honor a prominent resident, Charles Remsen, who had donated a new Presbyterian Church. Today, both names remain in use, each covering different areas of the community. The area close to the bay is called Remsenburg, and the more bucolic inland area is still called Speonk