On May 22, 1863, the U.S. War Department issued General Order Number 143, creating the Bureau of Colored Troops to recruit and organize regiments of African American soldiers. Frederick Douglass had been arguing for two years that the war could not be won without Black soldiers, and that the rights of Black citizens after the war depended on their having served. Lincoln's Emancipation Proclamation, effective January 1, 1863, had freed enslaved people in Confederate territory and authorized their enlistment. By war's end, the USCT made up roughly 10 percent of all Union soldiers. About 40,000 of them died, including 30,000 from disease - a far higher rate than the white regiments suffered, in part because Black soldiers were initially given worse food, worse equipment, and worse medical care. Twenty-five Medals of Honor went to African American soldiers and sailors during or shortly after the war. Many of those names - Powhatan Beaty, Christian Fleetwood, Robert Pinn, Milton Holland, Alexander Kelly - are on the wall at U Street.

The D.C. Commission on the Arts and Humanities commissioned the statue in 1993 from Ed Hamilton, an African American sculptor who had spent his career making public monuments in Louisville and around the South. The work depicts three USCT soldiers and one sailor on one side, and on the other a family they are leaving behind - a wife, an aged grandfather, and a young child looking up. The piece was completed in 1997 and dedicated on July 18, 1998. Around it the African American Civil War Memorial Freedom Foundation arranged the Wall of Honor: curved granite walls set with 166 burnished stainless steel plaques that hold the names of every USCT soldier and sailor, organized by regiment. The site was transferred to the National Park Service in 2004 and is now administered by National Mall and Memorial Parks. The memorial sits one block east of the U Street Metro station entrance and across the street from the Howard Theatre, in the heart of what was once called Black Broadway.

The museum that interprets the memorial opened in January 1999 in a small storefront at 1925 Vermont Avenue, directly across the street from the statue. In 2018 it moved to the historic Grimke School building at 1923 Vermont Avenue. Archibald Grimke, the school's namesake, was an African American lawyer, diplomat, and civil rights leader who served as the U.S. consul to the Dominican Republic in the 1890s and as president of the American Negro Academy in the early twentieth century. His mother was an enslaved woman; his father was the white plantation owner who fathered him. Archibald and his brother Francis - who became a leading Presbyterian minister - were both eventually recognized by their white aunts, the abolitionists Sarah and Angelina Grimke. The school named for Archibald - originally built in 1887 as the Phelps School and renamed in his honor in 1934 - has been preserved as part of a mixed-use redevelopment. The museum occupies the former gymnasium wing.

Among the museum's collections is the African American Civil War Memorial Registry, which holds the family trees of more than 2,000 documented descendants of men who served in the USCT and the segregated Navy. Living descendants can register their lineage and search the database for relatives. The registry is one of the only systematic genealogical projects focused specifically on African American Civil War veterans, a population whose service records were often dispersed, lost, or never properly maintained because of the racism of the bureau system that filed them. The exhibits inside the museum include photographs, original recruitment posters, replicas of period uniforms and weapons, and primary documents - pension applications, letters, regimental orders. The registry and the wall of names work together: a visitor can find a great-great-grandfather on the granite outside, then look up his service inside. Among the names on the wall: Robert Smalls, the enslaved harbor pilot who in May 1862 commandeered a Confederate steamer in Charleston Harbor and sailed it past the harbor batteries to the Union blockade, freeing himself and the fifteen other people aboard. Smalls later served in the U.S. Navy, then in the South Carolina state senate, and eventually in the United States House of Representatives. He is one of the lucky few whose name became famous. The other 209,144 names on the wall are also written there - and the wall is the same height for every one of them.