Stephanus van Cortlandt built his home on the site in 1671. When he retired to his Hudson River manor in 1700, the property passed to his son-in-law, Etienne "Stephen" DeLancey, a French Huguenot who married Van Cortlandt's daughter Anne. DeLancey built the current structure in 1719 using small yellow bricks imported from the Dutch Republic, creating a mansion that ranked among the finest in the province. His heirs sold it in 1762 to Samuel Fraunces, who converted the house into the Queen's Head Tavern, named for Queen Charlotte. Before long, the building became a meeting place for the Sons of Liberty. During the tea crisis sparked by the Tea Act of 1773, Patriots forced a British naval captain to publicly apologize at the tavern for attempting to bring tea to New York. Then, dressed as American Indians in the style of the Boston Tea Party, they dumped his cargo into New York Harbor.

Fraunces Tavern's most celebrated moment came after the fighting ended. A week after the British evacuation, Washington hosted an elaborate farewell dinner in the Long Room for his officers. After his famous words, he embraced each man individually. The scene became an iconic image of the Revolution's end -- a general voluntarily surrendering military power. But the tavern's role in the new republic was just beginning. When New York City became the seat of the Confederation Congress in January 1785, the departments of Foreign Affairs, Finance, and War set up offices in the building. It functioned as an early American government headquarters until Congress relocated to Philadelphia in 1790 under the Residence Act. The New York Chamber of Commerce had been founded at the tavern in 1768, and what would become NewYork-Presbyterian Hospital held its first board of governors meeting there in 1771.

The building endured fires beginning in 1832, changed hands repeatedly, and by 1900 was slated for demolition. The Daughters of the American Revolution led the effort to save it, convincing city officials to use their eminent domain powers to designate the building as a park -- the only legal mechanism available, since historic preservation laws did not yet exist. In 1904, the Sons of the Revolution acquired the property, funded largely by a bequest from Frederick Samuel Tallmadge, grandson of Benjamin Tallmadge, Washington's chief of intelligence. Architect William Mersereau completed a restoration in 1907 using the Philipse Manor House in Yonkers as a style guide. The result was controversial even then. Architects Norval White and Elliot Willensky later called it "a highly conjectural reconstruction -- not a restoration -- based on 'typical' buildings of 'the period,' parts of remaining walls, and a lot of guesswork." Guidebooks of the era, unbothered by such distinctions, called it "the most famous building in New York."

On January 24, 1975, a bomb exploded in the tavern's entrance hallway, killing four people and injuring more than fifty. The Fuerzas Armadas de Liberacion Nacional Puertorriquena (FALN), a Puerto Rican clandestine paramilitary organization, claimed responsibility. In a note found in a nearby phone booth, the group took credit for what it called a "specially detornated" -- their misspelling -- bomb aimed at "reactionary corporate executives." The device was roughly ten pounds of dynamite crammed into an attache case. The FALN said the bombing was retaliation for an explosion at a restaurant in Mayaguez, Puerto Rico, two weeks earlier that had killed three people. As of 2022, no one has been prosecuted for the attack. The tavern was repaired, and reopened.

Today, Fraunces Tavern occupies five interconnected buildings. The museum on the upper floors contains nine galleries, including the Long Room where Washington said goodbye, a collection of John Ward Dunsmore paintings depicting scenes from the Revolution, and a gallery of Washington portraits. The bar and restaurant operate on the lower floors, serving drinks and meals in the oldest surviving building in Manhattan. The site is listed on the National Register of Historic Places, designated a New York City landmark, and sits within a National Historic Landmark District. The block bounded by Pearl, Water, and Broad Streets and Coenties Slip has been designated a landmark district since 1978. Whether Mersereau's reconstruction faithfully represents the 1719 original is a question historians will continue to debate. What is not debatable is that the building has been in continuous use for over three hundred years -- a tavern that once poured drinks for revolutionaries and still pours them today.