Folk art in the 1920s was not yet a category academic museums respected. The Metropolitan Museum of Art and the Museum of Modern Art each accepted a portion of Abby Rockefeller's holdings, but the bulk needed a home of its own. The Ludwell-Paradise House in Williamsburg took the first loan in 1935 - works dating back to the 1720s, a span covering more than two centuries of unsigned American eyes looking at unsigned American things. After Abby died in 1948, her son David Rockefeller added another 54 objects she had given to the Met and to MoMA. By 1956 the gathered collection was outgrowing the colonial house that held it.

Nina Fletcher Little, a pioneer scholar of American decorative arts, advised the Rockefellers on the new museum. Her instinct was the opposite of grandeur. 'Lower the ceilings,' she suggested, and turn the interior into a series of rooms scaled to the houses these objects had originally inhabited. She did the first scholarly research on the collection, attributed paintings, and wrote its first catalogue. The Abby Aldrich Rockefeller Folk Art Collection opened in May 1957 in a two-story Georgian-Federal Revival brick building with a prominent oval garden, just outside Colonial Williamsburg's historic district at South England Street, next door to the Williamsburg Inn. In 1992 the museum added 19,000 square feet of single-story brick gallery with a planted wood-and-brick pergola and a fountain garden, designed by Roche-Dinkeloo.

In 2007 the folk art collection moved. It left South England Street and joined the DeWitt Wallace Decorative Arts Museum on Francis Street between Nassau and South Henry, near Merchants Square. The two museums now share an entrance below Colonial Williamsburg's restored Public Hospital of 1773 - an arrival sequence the official description calls 'notably circuitous,' meaning visitors descend beneath an eighteenth-century mental hospital to reach the eighteenth-century portraits inside. The new home gave folk art 10,400 square feet of exhibition space spread across eleven galleries. The old building with its oval garden, and the 1992 addition with its fountain, became part of the Spa of Colonial Williamsburg - the same brick walls Nina Little had designed scaled to domestic life, now scaled to massage tables.

In 2014 the Colonial Williamsburg Foundation announced a $40 million addition to expand the joint museum complex. The expansion completed in 2020, nearly tripling the gallery space and giving the folk art collection a dedicated street-level entrance for the first time since it left South England Street. Thomas N. Armstrong III - who had run the Whitney Museum of American Art in New York - served as one of the museum's most notable curators, bringing big-museum scholarship to a collection that began as one wealthy woman's contrarian taste in the 1920s. The museum now holds the deepest collection of American folk art anywhere: paintings by anonymous limners, painted furniture, carved birds and ship figureheads, samplers stitched by twelve-year-olds, and weathervanes that once swung above barns from Maine to Virginia.