The Zeughaus was commissioned by Elector Frederick III of Brandenburg in 1695 and designed by a succession of architects, with Andreas Schluter producing the masks of dying warriors that still ring the inner courtyard, the Schluterhof. It served as a working armory until the 1870s. The Hohenzollerns turned it into a military museum in 1875. The Nazis added their own propaganda exhibitions; the building was hit by Allied bombing during the war but largely survived. East Germany's leadership chose it in 1950 as the home of the Museum of German History, charged with presenting a Marxist-Leninist reading of the German past. After reunification on October 3, 1990, the entire collection of that East German museum was transferred to the new German Historical Museum, which had been founded three years earlier in West Berlin. The walls have heard arguments in five different ideological dialects.

The DHM was founded on October 28, 1987, on the 750th anniversary of Berlin. The driving force was West German Chancellor Helmut Kohl, who wanted a national museum that would present German history with what he called 'historical contextualization,' meaning that the Nazi period should be seen as one chapter in a long and largely admirable national story. The proposal was deeply controversial. Critics including Jurgen Habermas argued that Kohl was trying to relativize the Holocaust. The historians Kohl commissioned to write the founding memorandum, including Hartmut Boockmann, Eberhard Jackel, Hagen Schulze, and Michael Sturmer, produced a document that became the museum's intellectual charter. The fall of the Berlin Wall changed everything. The Italian architect Aldo Rossi had won the original design competition in 1988 for a new building near the Reichstag. After reunification, the museum took over the Zeughaus instead, and Rossi's design was shelved.

The Zeughaus underwent a comprehensive restoration between 1998 and 2003. Adjacent to it, the Chinese-American architect I. M. Pei was commissioned to design a modern exhibition hall for temporary shows. Pei's wing, completed in 2003, has an exposed glass-and-steel spiral staircase that has become an icon in its own right; it lets visitors climb four floors while looking out across the Spree to Museum Island. The structural engineering was by Leslie E. Robertson Associates, the firm that had engineered the World Trade Center. Pei was the same architect who designed the glass pyramid at the Louvre and the East Building of the National Gallery in Washington. His Berlin wing is intentionally subordinate to the Zeughaus, joined by an underground passage. The permanent exhibition reopened in the Zeughaus on June 2, 2006, with Chancellor Angela Merkel cutting the ribbon.

The collection spans roughly two thousand years and is divided across more than a dozen sections. The militaria collection of weapons and uniforms is the inheritance of the old Prussian armory; the everyday-life collection of toys, fashion, household goods, and product advertising holds about 155,000 objects. The poster collection includes the Hans Sachs collection, expropriated from the Sachs family by the Nazis in 1938 and returned to his heirs by court decision in 2012. The print collection holds about 100,000 items going back to the 16th century. The picture archive contains roughly half a million photographs. The library has over 225,000 volumes. The online database, the most extensive of any German museum, makes about 500,000 objects searchable, with digital photos for around 70 percent of them. Roughly 7,000 new objects enter the collection each year.

In 2015, in response to the wave of refugees arriving in Germany from Syria, Iraq, and Afghanistan, the museum launched a project called Multaka, the Arabic word for 'meeting point.' Specially trained guides, themselves often refugees, lead free tours in Arabic and Farsi for new arrivals. The conversations focus on cultural objects in the collection that connect to visitors' own heritage, including artifacts from across the Islamic world. The project has spread to museums across Europe, with a network now spanning the United Kingdom, Italy, Greece, and Switzerland. The Zeughaus has been closed for renovation since June 2021, with a complete redesign of the permanent exhibition; reopening is planned for late 2025. The Pei wing remains open with rotating temporary exhibitions. The Foundation for Flight, Expulsion, and Reconciliation, which documents the postwar expulsion of Germans from Eastern Europe, sits under the museum's institutional umbrella since 2009.