The road to opening was slow. In 1924 the North Carolina State Art Society formed to push for a museum. In 1929, seventy-five paintings went on display in temporary exhibition spaces in the state Agriculture Building in Raleigh. The collection moved to the former Supreme Court building in 1939. After the 1947 appropriation, the museum needed a permanent home, and the state gave it the renovated Highway Division Building on Morgan Street. NCMA opened there on April 6, 1956. W. R. Valentiner, a noted art historian, became its first director. Alice Willson Broughton, the former First Lady of North Carolina, had helped procure the original funding. In 1961 the state legislature separated the museum from the Art Society and made it a state agency.

By 1967, the museum had outgrown Morgan Street. The state bought a site on Blue Ridge Road in west Raleigh. Edward Durell Stone, the architect of the Kennedy Center in Washington and the U.S. Embassy in New Delhi, was hired with North Carolina firm Holloway-Reeves to design the new building. Stone treated the site as a meditation on the square: every space, every wing, every garden was generated by manipulating one geometric unit. It was his last major design. He died in 1978, before the museum opened in 1983. After his death, the exterior was changed from white marble to red brick. The building still uses Stone's geometry. The opening exhibition, Renaissance to Impressionism, drew crowds that had been waiting for years.

The collection spans antiquity to the present. The European galleries hold a Titian, a Raphael, paintings by Murillo, Monet, Pissarro, Boudin, and Millet, and more than thirty bronzes by Rodin, some of them set into a dedicated Rodin Garden outside. The Italian Renaissance is anchored by Cima da Conegliano's Virgin and Child in a Landscape. The Northern European rooms include a Flemish baroque kunstkamer and works by Rubens, van Dyck, Jordaens, Jan Brueghel the Elder, Jan Steen, and Jacob van Ruisdael. The Egyptian collection is small, just thirty-eight artifacts, but it spans the Predynastic to the Roman period, including the painted coffins of Djed Mut and Amunred and a canopic jar that once held the mummified liver of a man named Qeny. The Judaic gallery is one of only two in any American art museum. The contemporary collection includes Bill Viola video, Kehinde Wiley paintings, and the giant white Jaume Plensa Spiegel sculptures, installed in the garden in 2010.

The 127,000-square-foot West Building, opened in 2010, received an international design award for energy efficiency. But the building most people remember at NCMA is not really a building. It is the 164-acre Museum Park around it, one of the largest museum art parks in North America. Two miles of trails wind through fields, woodlands, and creek beds, past site-specific works by Thomas Sayre, the late Vollis Simpson, Martha Jackson-Jarvis, and Ledelle Moe, and through Chris Drury's Cloud Chamber for the Trees and Sky, a camera obscura that turns the sky overhead into a slow projection on a curved wall. Henry Moore bronzes anchor the West Building gardens. Barbara Kruger's monumental Picture This sits in the amphitheater, currently under construction and dark through 2026. When the amphitheater reopens, the outdoor concert series, paused since 2024, will return to one of the most ambitious museum landscapes in the country.