n the middle of the 19th century, the island was called Lesser Minneford Island. The island was part of the (0.2 sq miles) property purchased by Thomas Pell from the local Native Americans in 1654. In February 1869, New York City purchased the island from Edward Hunter of the Bronx for $75,000.There are several versions of the origin of the island's name. In one, British cartographers named it "Heart Island" in 1775, due to its organ-like shape, but the middle letter was dropped shortly thereafter.
Others sources indicate that "hart" refers to the Middle English word for "stag." One version of this theory is that the island was given the name when it was used as a game preserve. Another version holds that it was named in reference to deer that migrated from the mainland during periods when ice covered that part of Long Island Sound. A passage in William Styron's novel Lie Down in Darkness describes the island as occupied by a lone deer shot by a hunter with a row boat. Styron provides a vivid description of the public burials following World War II including the handling of remains from re-excavated graves.