In 1831, a small group of Episcopalians complained to Bishop Richard Channing Moore that Richmond's two existing Episcopal churches were on the wrong side of the growing capital. Both stood east of the action; the new commercial energy was building west. By 1835, they had bought a lot at 5th and Marshall Streets and put up a Sunday School. Rev. Adam Empie — formerly rector of Bruton Parish in Williamsburg and president of the College of William and Mary — arrived in 1837 to lead them. The cornerstone went down on April 2, 1838. Bishop Moore consecrated the finished sanctuary on June 23, 1839. The church stood about a block from Richmond's slave market, an uncomfortable geography the parish would carry with it for decades. Like his predecessors, Rev. Joshua Peterkin owned enslaved people. The original church built a gallery for African-American congregants, and Peterkin reestablished a school to teach African Americans first started by Empie in 1845. The contradictions of Richmond Episcopalianism lived inside these walls.

By 1861, St. James's was Richmond's largest Episcopal congregation. The Civil War took a heavy toll. General J.E.B. Stuart, a parishioner, was killed at the Battle of Yellow Tavern in May 1864. Capt. O. Jennings Wise — son of former Governor Henry A. Wise, who had famously refused to spare John Brown's life after the raid on Harpers Ferry — died at Roanoke Island in 1862. The most extraordinary story belonged to Sally Tompkins. She opened a private hospital to nurse Confederate casualties, and when the army outlawed private hospitals, she accepted a military commission so her work could continue — becoming the Confederate Army's only female officer. Of the 1,334 soldiers admitted to her care by 1865, only 73 died. The Evacuation Fire of April 1865 spared the building but ruined many parishioners. From that wreckage, the Peterkins helped found Sheltering Arms Hospital for those who could not afford care.

By the early twentieth century, Shockoe Hill had emptied out and the wealthy were moving to the Fan District. Two prominent clergymen turned down the rectorship rather than serve in what had become an isolated location, made worse by a new streetcar line that rattled services. Rev. William Meade Clark finally pushed through the move. The cornerstone of the current building near Stuart Circle and Monument Avenue was laid on May 17, 1912. Richmond architects Noland and Baskervill — who also designed Temple Beth Ahabah — drew the new church after both its predecessor and Sir James Gibbs's St. Martin-in-the-Fields in London. Consecration waited until 1918, until the building debt was fully paid. In 1926, vestry member Henry Lee Valentine donated the chimes whose daily peals soon became a Fan District tradition.

Then came July 13, 1994. Lightning struck the church, the altar burned, the nave burned, the roof collapsed. The four walls held. Richmond's firefighters performed a small miracle that night: they saved all ten of the nave's stained glass windows, including four panels by Louis Comfort Tiffany. After the fire, the parish rebuilt under Rev. Robert Trache, and the new sanctuary was rededicated in 1997. It contains C. B. Fisk's Opus 112, considered one of the finest pipe organs in North America. The congregation also revised the parish seal, replacing one blank quadrant with tongues of flame — a nod to the Holy Spirit and a quiet acknowledgment of the fire itself. Four pews from the original survived; the rest are reproductions made in New Holland, Pennsylvania. In January 2026, Rev. Beth Franklin will be welcomed as the parish's 15th rector, continuing a line that runs back nearly two centuries.

Above the altar runs the motto attributed to James the Just: "Be ye doers of the Word and not hearers only." The parish seal mixes traditions — three scallop shells of St. James the Greater, the pelican pecking its breast as a symbol of sacrifice, a fish and loaf, and now the post-fire flames. The 1895 brass pulpit, donated by the women of the parish in memory of Rev. Joshua Peterkin, was disparaged at the time by vestryman and sculptor Edward Valentine, who called it "vulgar." It survives anyway. In 1951, the United Daughters of the Confederacy donated a window memorializing General Stuart; a century after her death, another window was added for Capt. Sally Tompkins. The 1986 St. James Children's Center became the only preschool in Richmond welcoming children with special needs. Layer by layer, the church accumulates its history without erasing any of it.