The museum traces its origins to the Panama-California Exposition of 1915, which San Diego organized to celebrate the opening of the Panama Canal. The central exhibit of the exposition, called 'The Story of Man through the Ages,' was assembled under archaeologist Edgar Lee Hewett, who organized expeditions to gather pre-Columbian pottery from the American Southwest and artifacts from Mayan sites in Guatemala. Materials also came from expeditions led by Smithsonian anthropologist Aleš Hrdlička, who gathered specimens from Africa, Siberia, Alaska, and Southeast Asia.

When the exposition ended, a group of citizens led by George Marston formed the San Diego Museum Association to retain the collection and convert it into a permanent museum. The California Building and its landmark tower became the museum's permanent home — the same building that Welles would later use as Xanadu.

The museum's permanent holdings are substantial and eclectic: more than 100,000 documented ethnographic items, 300,000 archaeological pieces, and 25,000 photographic images. The focus is on the pre-Columbian history of the western Americas, with materials from Native American cultures of Southern California and Mesoamerican civilizations.

Among the most unusual objects are seven painted wooden coffins from ancient Egypt, donated by Ellen Browning Scripps and the Egypt Exploration Society. One of these is a Ptolemaic child's coffin — only six others are known to exist anywhere in the world. The collection also includes materials from the city of Amarna, the Egyptian capital founded by the pharaoh Akhenaten. These Egyptian holdings exist in a San Diego museum largely because of the philanthropic generosity of the Scripps family, whose name appears throughout the city's cultural and scientific institutions.

The California Building's tower was closed to the public for nearly eighty years before reopening in 2015 for the centennial of the Panama-California Exposition. The tower contains a carillon whose quarterly-hour chimes can be heard across much of Balboa Park — a sound that has become part of the park's ambient identity even for visitors who never think about where it comes from.

The tower's reopening allowed visitors to see the park from an elevation that had been unavailable for generations, and to understand the building's original function as a landmark visible from a distance — a signal that something significant lay within.

In 1942, the museum changed its name to the Museum of Man, reflecting the anthropological framing of its era. In 1978, 'San Diego' was added. Both names carried the weight of a particular historical moment — the post-war confidence in universal human science, the localism of a city growing into its own cultural identity.

On August 2, 2020, in the context of a global wave of institutional reckoning with questions of representation and colonial history, the museum changed its name again — to the Museum of Us. The new name was chosen to signal a shift toward inclusiveness and decolonization in how the museum presents and interprets its collections. The building that inspired Orson Welles still stands unchanged. The institution inside it is still working through what it means to hold these particular objects in this particular place.