The Legion of Honor was the vision of Alma de Bretteville Spreckels, wife of sugar magnate Adolph B. Spreckels. The building is a full-scale replica, by architects George Applegarth and Henri Guillaume, of the French Pavilion at the 1915 Panama-Pacific International Exposition, which was itself a three-quarter-scale version of the Palais de la Legion d'Honneur in Paris. At the close of the exposition, the French government granted Spreckels permission to construct a permanent copy. World War I delayed groundbreaking until 1921. The museum was dedicated as a memorial to California soldiers killed in the war. The land on which it stands was once the Golden Gate Cemetery, established in 1870 and closed in 1909, which had held approximately 29,000 remains, including a Chinese burial ground and a potter's field. The dead were relocated. The palace went up.

The Legion's collection spans more than six millennia, from ancient Egyptian carved wood figures to Impressionist canvases. Its most distinguished holdings are the sculptures of Auguste Rodin, whose Thinker stands in the Court of Honor. El Greco, Titian, Rubens, Rembrandt, Monet, Degas, Renoir, Cezanne, and Van Gogh are all represented. The Achenbach Foundation for Graphic Arts holds more than 90,000 works on paper, making it the largest such repository in the western United States. The collection of European decorative arts includes a gilded Spanish ceiling from around 1500 and the Salon Dore from the Hotel de La Tremoille in Paris, said to be the only complete pre-Revolutionary Parisian salon on display anywhere in the world. Between 1992 and 1995, a major renovation added 35,000 square feet underground, piercing the Court of Honor with a pyramidal skylight that quotes the Louvre Pyramid in miniature.

In 1924, John D. Spreckels commissioned the Ernest M. Skinner Company of Boston to build a symphonic organ for the museum. Its 4,526 pipes are completely invisible, hidden behind a trompe l'oeil ceiling painted to resemble a marble apse. Every Saturday at four in the afternoon, the organ fills the gallery with sound, and visitors who did not know the pipes were there look up, startled. The 316-seat Gunn Theater downstairs hosts San Francisco Symphony chamber concerts beneath a ceiling mural by Spanish artist Julio Vila y Prades depicting the Apotheosis of the California Soldier. It is a building that keeps revealing layers, visual and acoustic, historical and emotional, the longer you spend inside it.

The museum occupies an elevated site in Lincoln Park in San Francisco's northwest corner. The Golden Gate Bridge is visible from the grounds. The downtown skyline shimmers in the distance. George Segal's Holocaust Memorial, a sculptural group of white-painted bronze figures installed in 1984, stands off the northwest corner of the grounds, a stark counterpoint to the French neoclassicism of the building. In 2023, President Joe Biden held an APEC state dinner at the Legion of Honor for world leaders from member economies, a reminder that this replica palace on a cemetery on a cliff continues to serve as one of the most dramatic settings in American public life.