In 1847, two prominent Richmond citizens, Joshua J. Fry and William H. Haxall, traveled to Boston and visited Mount Auburn Cemetery. The pioneering rural cemetery, with its parklike grounds and winding paths, deeply impressed them. They returned to Richmond determined to build something similar. Fry, Haxall, and forty other local citizens purchased 42 acres of land that had once belonged to the Harvie family, who had bought it from the estate of William Byrd III, the indebted planter who had subdivided his Belvidere property. They hired landscape architect John Notman to design the grounds. Notman proposed the name Hollywood, after the holly trees that grew across the property. The cemetery was dedicated in 1849. It became so popular that by the mid-1850s, the city of Richmond ran an omnibus to transport visitors there every afternoon. A streetcar line followed in the 1860s. What began as a burial ground quickly became one of Richmond's essential public landscapes.

Hollywood Cemetery is one of only three places in the United States where two U.S. presidents are buried. The others are Arlington National Cemetery and United First Parish Church in Quincy, Massachusetts. President James Monroe died in New York City in 1831 and was originally interred in Marble Cemetery in Manhattan. Virginia later petitioned to have his remains brought home. The Gothic Revival monument designed by Albert Lybrock, an ornate ironwork cage surrounding a simple granite sarcophagus, was built in the Presidents Circle section and dedicated by Governor Henry A. Wise on July 5, 1858. The monument was designated a National Historic Landmark in 1971. President John Tyler was buried in the Presidents Circle in 1862, but Washington refused to acknowledge his death because of his allegiance to the Confederacy. Jefferson Davis himself escorted the burial ceremony. Congress did not dedicate a monument at Tyler's grave until 1915, more than half a century after his death.

Richmond served as the capital of the Confederate States of America, and Hollywood Cemetery became the resting place for the Confederacy's dead at every level. The cemetery holds 25 Confederate generals, including J.E.B. Stuart, Fitzhugh Lee, and George Pickett. Secretary of War James A. Seddon is here. And Confederate President Jefferson Davis, who died in 1889 and was initially interred in Metairie Cemetery in New Orleans, was eventually reinterred at Hollywood. But the most striking memorial belongs to the enlisted men. In 1869, a 90-foot granite pyramid designed by Charles H. Dimmock was erected on a hilltop as a memorial to more than 11,000 Confederate soldiers buried in the cemetery. Built from roughly cut James River granite blocks, the pyramid bears a Latin inscription that translates to "In eternal memory of those who stood for God and Country." The Hollywood Memorial Association placed it on the highest ground so it would be the first thing visitors see when entering the cemetery. It remains the dominant feature of the skyline within the grounds.

Hollywood Cemetery has accumulated layers of local legend. The most famous involves William Wortham Pool, whose grave in the cemetery became attached to the urban legend of the Richmond Vampire. The story traces back to the 1925 Church Hill Tunnel collapse, when witnesses claimed a pale, bloodied figure emerged from the wreckage and fled to the cemetery. Pool's mausoleum became the lore's anchor point, and the tale has persisted for a century. On May 31, 1866, Hollywood Cemetery hosted its first Confederate Memorial Day, drawing over 20,000 people. The Gothic Revival entrance tower, designed to resemble a ruined medieval fortification, was built in 1876. The cemetery expanded in 1877 with thirty-three additional acres along the James River, and it was added to the National Register of Historic Places on November 12, 1969. In 2020, the board of directors quietly banned Confederate flags from the grounds. Today, students from nearby Virginia Commonwealth University walk the paths alongside tourists and history scholars, making Hollywood Cemetery as much a park for the living as a memorial to the dead.