The museum exists because Judge John Barton Payne could not bear to keep his paintings to himself. In 1919, he donated fifty paintings to the Commonwealth of Virginia in memory of his late second wife Jennie Byrd Bryan Payne and his mother Elizabeth Barton Payne. The state had nowhere to put them. During the Great Depression, Payne partnered with Virginia Governor John Garland Pollard to secure funding through Franklin Roosevelt's Works Progress Administration, augmenting state funds to build a proper museum. The site chosen sat on Richmond's Boulevard next to a contiguous six-block tract that served as a veterans home for Confederate soldiers and their families — an uncomfortable adjacency the museum has grappled with explicitly in the twenty-first century. The main building, designed by Peebles and Ferguson Architects of Norfolk in a Georgian Revival style influenced by Inigo Jones and Christopher Wren, began construction in 1934. Opening day was January 16, 1936. In 1951, the museum bought an abstract painting called Chimneys by a 20-year-old art student named Benjamin Leroy Wigfall at the historically Black Hampton Institute — the youngest artist whose work had ever been purchased by the museum, and one of its earliest acquisitions of a painting by a Black artist.

Leslie Cheek Jr. arrived as director in 1948 and stayed until 1968. The New York Times obituary credited him with transforming the VMFA "from a small local gallery to a nationally known cultural center." His innovations were practical and theatrical at once. In 1953 he invented the Artmobile — a tractor-trailer of art that drove into rural Virginia counties at a time when most rural Americans had never set foot in a museum. In 1960, he became the first art museum director in the United States to introduce evening hours. He hung velvet drapery around the Fabergé eggs. He gave the Egyptian wing a tomb-like setting and piped in music to set moods. Under Cheek, the 1954 first addition went up with funding from Paul Mellon. Cheek also pushed through construction of a 500-seat theater. Through the 1960s it hosted a community theater company; in 1969 Keith Fowler founded Richmond's first resident Actors Equity company there. Critic Clive Barnes praised Fowler's 1973 Macbeth in the New York Times as "splendidly vigorous." The theater struggled financially through the 1990s and closed in 2002. Renovated and reopened in 2011, it now bears Cheek's name.

Three families shaped the collection. Lillian Pratt — wife of John Lee Pratt of Fredericksburg — donated the Pratt Fabergé collection, the largest holding of Fabergé outside Russia. It includes five Imperial Easter Eggs: Rock Crystal (1896), Pelican (1898), Peter the Great (1903), Tsarevich (1912), and Red Cross with Imperial Portraits (1915). Each one was made by Peter Carl Fabergé for the Tsar to give as an Easter gift. Paul Mellon — son of banker Andrew Mellon — gave the museum a stunning collection of British sporting art in 1983, plus French Impressionist and Post-Impressionist works. When Mellon died in 1999 he bequeathed five paintings by George Stubbs and additional French and British works. The Mellon Galleries closed in 2018 for renovation and reopened with new context. Sydney and Frances Lewis built a Modern and Contemporary collection of more than 1,200 works in the mid- to late twentieth century — many acquired through their Richmond company, Best Products, which traded appliances and electronics for artwork. The Lewises did not just collect artists; they befriended them. Their collection forms the bedrock of VMFA's modern holdings.

In 2019, the museum installed a monumental Kehinde Wiley bronze sculpture on its front lawn facing Arthur Ashe Boulevard. The work, called Rumors of War, depicts a young Black man in dreadlocks, ripped jeans, and a hoodie astride a powerfully muscled horse — a direct response to the Confederate equestrian statues that lined Monument Avenue a few blocks away. Wiley had visited Richmond in 2016 for a retrospective at the VMFA and conceived the piece after seeing the Monument Avenue statues. Rumors of War debuted in Times Square before being installed in Richmond as a permanent gift to the museum. Within two years, most of the Confederate statues it was answering had been removed. The sculpture stands. In 2010, the museum had completed a $150 million expansion that added 165,000 square feet of gallery space — the McGlothlin Wing, designed by London-based Rick Mather with Richmond's SMBW Architects. The expansion won a 2011 RIBA International Award. A further $190 million expansion announced in 2021 will add gallery space for African, African American, and 21st-century art.

The encyclopedic collection covers African, American, Ancient, Art Nouveau and Art Deco, English silver, European, Fabergé, Modern and Contemporary, South Asian, and East Asian art. The African collection has grown from 250 objects in the mid-1990s to roughly 500 today, with strengths in Kuba, Akan, Yoruba, Kongo, and Mali traditions. The Ancient galleries include "Tjeby," one of two Egyptian mummies in Richmond (the other lives at the University of Richmond). Art Nouveau holdings include René Lalique, Hector Guimard, Émile Gallé, Louis Comfort Tiffany, Charles Rennie Mackintosh, and Frank Lloyd Wright. In 2022 the museum acquired Portrait of a Creole Woman with Madras Tignon at auction for $948,000, deepening its representation of the colonial Americas. The VMFA Fellowship Program, established in 1940, has awarded over $5 million in grants to Virginia artists. Notable recipients include filmmaker Vince Gilligan and photographer Emmet Gowin. In 2004, the museum returned a sixteenth-century French painting that had been stolen by the Gestapo from Jewish Austrian collector Julius Priester in 1944.