John Armfield ran the Alexandria end of the operation. He lived in the building itself, purchasing enslaved people through an extensive network of agents across Virginia and Maryland and through advertisements placed in regional newspapers soliciting sellers. Behind the respectable brick facade stood a compound that occupied half a city block, with yards enclosed by high brick walls where human beings waited in pens for their forced journey south. Armfield shipped his purchases to his partner Isaac Franklin in New Orleans and Natchez, Mississippi, using two company-owned ships, the Uncas and the Tribune. In peak years during the early 1830s, the firm bought and sold between 1,000 and 2,000 people annually. Some were marched overland in coffles -- groups of dozens chained together walking a thousand miles to the Deep South. The operation was methodical, efficient, and staggeringly profitable.

Franklin retired in 1836, wealthy beyond measure, and the building passed through a succession of slave dealers. The last to operate from the site was James H. Birch, the same trader who paid accomplices to kidnap Solomon Northup, a free Black man from New York, in Washington, D.C., in 1841. Birch fabricated a false identity for Northup and shipped him to a partner in New Orleans, beginning the twelve years of enslavement that Northup later documented in his memoir, Twelve Years a Slave. The connection is a reminder that the slave trade operating from this building did not merely traffic in people already held in bondage; it also consumed free citizens. Built around 1812 as a private residence for Brigadier General Robert Young, the building had been converted to serve commerce in human lives within two decades of its construction.

When Union forces occupied Alexandria on May 24, 1861, they seized the slave trading compound and repurposed it as a military prison. It later served as L'Ouverture Hospital, treating wounded soldiers, and as barracks for contraband -- enslaved people who had escaped to Union lines. After the war, most of the compound's structures were demolished. Only the original brick building survived, its connection to the slave trade gradually fading from public memory. The structure was subdivided into apartments, its exterior renovated in a Second Empire style that obscured its Federal-era origins. For much of the twentieth century, the building's history went largely unacknowledged. A 1999 book, Lies Across America, noted how the site had been overlooked by the city's extensive historical tourism.

The Northern Virginia Urban League acquired the property in 1997 and began the work of restoration and remembrance, establishing the Freedom House Museum in the basement where enslaved people had once been held. The City of Alexandria purchased the building in March 2020, and after extensive renovation, reopened the museum in June 2022 with three exhibits documenting the Black experience in Alexandria and the broader story of American slavery. In 2025, the exterior was restored to its original Federal appearance, stripping away the later Second Empire alterations and returning the building to something closer to what it looked like when Armfield first set up shop. Today, the Freedom House Museum stands as one of the few surviving structures in the United States directly associated with the domestic slave trade. The restored brick facade no longer hides what happened here. It confronts it.