The Smithsonian Institution was founded in 1846 with a Congressional directive that it include 'a gallery of art.' For its first half-century, the institution's first secretary Joseph Henry largely ignored that instruction in favor of scientific research. An 1865 fire in the Smithsonian Castle destroyed much of what little art had been gathered, and the surviving pieces were farmed out to the Library of Congress and the Corcoran for decades. The collection came back in 1896. In 1906 the Smithsonian began calling it the National Gallery of Art. In 1937 Andrew Mellon offered the federal government a much larger collection on the condition that his new institution get the name, and the Smithsonian's gallery was renamed the National Collection of Fine Arts. In 1980 it became the National Museum of American Art. In 2000 it changed names again to the Smithsonian American Art Museum, mainly so the traveling exhibitions could trade on the Smithsonian brand. The current name has lasted a quarter-century, and the institution has stopped renaming itself.

The home Congress finally gave the museum in 1958 is one of the most important antebellum buildings in Washington. Construction began in 1836 under Robert Mills, the architect of the Washington Monument and the Treasury Building. The Greek Revival design centered on a south portico modeled on the Parthenon. The Patent Office moved out in 1932; the Civil Service Commission moved in; and by the late 1950s the building was scheduled for demolition until preservationists persuaded Congress to repurpose it. The Smithsonian got the northern half for art, the southern half for what would become the National Portrait Gallery. Renovation began in 1964. The 2000-to-2006 second renovation was supposed to take three years and 60 million dollars and ended up taking six years and 283 million. It restored the Parthenon porticos and the curving double staircase, added the Robert and Arlene Kogod Courtyard (a translucent canopy by Foster and Partners stretched over the central courtyard), and built the Lunder Conservation Center - the country's first art-conservation laboratory designed for public viewing. The whole reopened on July 1, 2006, jointly rebranded as the Donald W. Reynolds Center for American Art and Portraiture.

SAAM holds work by more than 7,000 American artists - one of the most comprehensive collections of art made in the United States from the colonial period to the present. It has the world's largest collection of New Deal art, the murals and sculptures and easel paintings commissioned by the WPA's Federal Art Project between 1935 and 1943. Its Gilded Age holdings include John Singer Sargent and Albert Pinkham Ryder. Its frontier landscapes by Albert Bierstadt and Thomas Moran helped sell the idea of the West to Eastern audiences in the 1860s and 1870s. Edward Hopper hangs near Georgia O'Keeffe and Winslow Homer. The folk-art holdings - which include the work of self-taught artists like Bill Traylor, the formerly enslaved Alabama farmhand who began drawing at the age of 85 and produced more than 1,500 surviving works in three years - are the deepest in the country. Edmonia Lewis, the African American and Ojibwe sculptor who carved Death of Cleopatra in Rome in 1876, is represented. So are Nam June Paik, Jenny Holzer, and the Korean-American video artists who pulled American art into late-twentieth-century media.

On the third and fourth floors of the museum sits the Luce Foundation Center, opened with the 2006 reopening - the first visible art storage facility in Washington. Visitors can browse 3,300 paintings, sculptures, folk-art objects, and crafts in 64 secure glass cases, quadrupling the amount of the permanent collection on view at any one time. Paintings hang densely on screens, salon-style. Folk-art carvings sit on open shelves. The Luce Center treats storage as exhibition: instead of curating one canonical selection per gallery, it shows the collection's true breadth. Below it, the Lunder Conservation Center lets visitors watch conservators work through tall glass walls - five labs equipped to treat paintings, prints, drawings, photographs, sculpture, folk art, contemporary craft, decorative arts, and frames. Both centers were designed to let museum visitors see what museums normally hide.

When the NCFA opened in 1968 the area around the Old Patent Office was, in the museum's own later phrasing, bleak and lonely. The 1968 riots had emptied F Street. Attendance stayed weak for three decades. The Pennsylvania Avenue Development Corporation began rebuilding F and G Streets in the 1980s. In 1997 the MCI Center (now Capital One Arena) opened directly across the street, bringing 18,000 hockey and basketball fans to the museum's doorstep on game nights. Restaurants followed, then condominiums, then the rebranded Penn Quarter neighborhood. By 2022 SAAM was drawing 1.1 million visitors a year, ranking seventh on the list of most-visited museums in the United States. The Renwick Gallery on Pennsylvania Avenue, SAAM's branch since 1972, doubles the museum's footprint. Together they hold the most inclusive collection of American art on Earth - assembled across two centuries of institutional renaming, fire loss, displacement, and the slow recovery of a neighborhood whose history the museum is partly a record of.