The Sackler Gallery began with a visit by Japanese Prime Minister Masayoshi Ōhira to the Freer Gallery of Art in 1979. The Freer, opened on the Mall in 1923 and named for Detroit railroad-car manufacturer Charles Lang Freer, held one of the great American collections of Asian art. But Freer's bequest had legally restricted the museum: the collection could not be lent, expanded, or shown outside Washington. The Smithsonian needed a complementary museum to allow loans and broader collecting. During his visit, Ōhira announced that Japan would donate $1 million to help build an annex. The U.S. Senate approved $500,000 in matching funds in June 1979. Congress finally appropriated $960,000 for construction in December 1981 - the first unrestricted federal contribution to the project. In 1982 the pharmaceutical executive and collector Arthur M. Sackler donated approximately 1,000 Asian objects from his private collection, valued at $50 million, along with $4 million in cash. The gallery was named for him. Construction began on June 21, 1983, with Vice President George H.W. Bush, Chief Justice Warren Burger, and Smithsonian Secretary S. Dillon Ripley turning the first shovels of earth.

Carlhian's design solved a difficult site problem. The Smithsonian's leadership wanted the new Asian and African art museums to share a campus on the South Quadrangle behind the Castle, but the surface space was already occupied by the historic Haupt Garden. The solution was to put the museums almost entirely underground - the Sackler Gallery, the National Museum of African Art, the S. Dillon Ripley Center for international scholarship, and connecting tunnels to the Freer. The buildings share aesthetic vocabulary: pink and gray granite drawn from the Smithsonian Castle, the Arts and Industries Building, and the Freer; geometric forms that echo each historic neighbor; entrance pavilions kept low to the ground. The Sackler pavilion's exterior decoration borrows from Islamic art - appropriate to a collection rich in Persian and Mughal manuscripts. The gallery opened on September 28, 1987. Mayor Marion Barry declared the day Smithsonian Institution Day. Arthur Sackler had died four months earlier and never saw the museum named for him.

Sackler's founding gift included Chinese ritual bronzes, Buddhist sculpture, Iranian and Mughal manuscripts, Indian paintings, Japanese woodblock prints, and ceramics from across Asia. The museum has expanded the collection steadily since. In 1986 the Sackler acquired the Vever Collection - an extraordinary group of Persian and Islamic manuscripts and miniatures collected by the French jeweler Henri Vever between 1900 and 1943. When curators Glenn Lowry and Milo Beach examined the Vever materials after acquisition, they were the first scholars in forty years to see them; the collection had been kept private and hidden during and after the Second World War. Subsequent acquisitions have included contemporary Japanese ceramics, photography of the Empress Dowager Cixi by Yu Xunling, and the 1997 loan exhibition of the Padshahnama Mughal manuscript from the Royal Library at Windsor Castle. Since 2019 the Sackler and Freer have operated jointly as the National Museum of Asian Art. The combined institution houses the largest Asian art research library in the United States and one of the leading conservation labs for Asian paintings - a tradition that goes back to 1932, when the Freer first hired a full-time Japanese restorer.

In 2018, protesters began appearing in the Sackler Gallery's lobby. The Sackler family's pharmaceutical company, Purdue Pharma, had aggressively marketed the opioid OxyContin starting in the late 1990s, contributing to an addiction epidemic that has killed hundreds of thousands of Americans. Arthur Sackler had died in 1987, two years before OxyContin was developed, and his branch of the family had been bought out before Purdue began the marketing campaign. His widow and supporters argued strongly that his philanthropic legacy should not be conflated with the actions of his brothers' descendants. The protesters argued that the name had become indistinguishable from the harm done. In 2019 the Metropolitan Museum of Art, the Louvre, and several other major institutions began removing the Sackler name from galleries funded by the family. The Smithsonian initially declined a full name removal. In December 2019 the institution rebranded both galleries under the public-facing name National Museum of Asian Art, and in April 2023 the Board of Regents formally endorsed that identity - subordinating the donor's name to the institutional one while leaving the legal building name unchanged. The art remains the same. The objects have outlived earlier benefactors and will outlive this one. The Persian miniatures are still in the dim galleries below the boxwoods, where the light is low and the air is climate-controlled, and the ink that Bihzad of Herat laid down five centuries ago is still legible.