The Confederacy chose an abandoned twenty-year-old cotton mill near the railroad line as its Salisbury prison site. In the war's early years, conditions were manageable. Prisoners received adequate food, moved freely within the compound, and organized recreational activities. The baseball game Boetticher captured in his drawing took place on July 4, 1862, a quietly defiant celebration of Independence Day behind enemy lines. His lithograph, later reproduced from the original watercolor, became one of the first artworks showing America's emerging national pastime. For a brief window, Salisbury was a prison where men could still play.

By October 1864, everything changed. The prison's population surged to 5,000, then quickly to 10,000 -- five times the population of Salisbury itself, which had only 2,000 residents and was still the fourth-largest town in North Carolina. Local residents grew alarmed by the sheer numbers and the diseases that spread from the overcrowded compound. The death rate climbed from 2 percent to 28 percent as malnutrition, poor sanitation, and epidemics ravaged the prisoners. Confederate General T.W. Hall's own report documented the grim arithmetic: between October 5, 1864, and February 17, 1865, some 10,321 prisoners arrived, and 2,918 died at the hospital alone. The Confederates resorted to mass graves. Among the dead was an eighteen-year-old private named Rupert Vincent of the 3rd New Hampshire Infantry -- in reality Robert A. Livingstone, the eldest son of the famous Scottish explorer Dr. David Livingstone, who had enlisted under a false name and died of wounds sustained in a prison riot.

In February 1865, as conditions grew untenable, the Confederates began transferring prisoners elsewhere. Some 3,729 were marched to Greensboro and sent by train to Wilmington, while 1,420 others went to Richmond. When Union General George Stoneman reached Salisbury that spring, the prison had been emptied and converted to a supply depot. Stoneman ordered the prison structures burned and a wooden fence erected around the mass graves. After the war, the cemetery was officially designated a National Cemetery for Union burials, and remains of Union troops from surrounding areas were transferred in. The wooden fence gave way to a stone wall. In 1908, the state of Maine erected a granite monument topped with a soldier's statue; Pennsylvania followed with its own monument in 1909.

The monument to unknown soldiers at Salisbury bears a number: 11,700 dead, an estimate made in 1869 by Brevet General L. Thomas after two trenches were opened. But later research by historian Louis A. Brown placed the actual count closer to 3,800. Mark Hughes of Kings Mountain, North Carolina, has campaigned for decades to correct the monument and to add individual grave markers for the roughly 3,500 men whose identities can be traced through an 1868 Roll of Honor. Federal law requires a marker for anyone buried in a national cemetery, yet as of 2014, the Department of Veterans Affairs had not planned to change the monument or add individual markers. An interpretive panel installed in 2009 now explains the discrepancy, a compromise that satisfies the letter of transparency without altering the stone.

The cemetery has continued to expand across its more than 160 years of service. By the 1990s, with roughly 5,800 burials across 12.5 acres, administrators predicted the original grounds would run out of space by 1999. A new annex was opened, and in 2011 construction began on a columbarium with capacity for 1,000, along with 2,400 pre-placed crypts that allowed 1,500 burials per acre. In January 2020, the Rowan County YMCA donated more than 30 acres to extend the annex's capacity through 2065. The original cemetery, with about 7,000 markers, closed to new burials on Memorial Day 2012. The annex remains North Carolina's only open national cemetery, still receiving the state's veterans into ground that first opened for the Civil War dead.