Born in Hamburg on 22 July 1887, Hertz was the nephew of Heinrich Hertz, the physicist whose name became the unit of frequency. In 1913 he took up a research assistant post at the Physics Institute of the University of Berlin. The following year, working with James Franck, he performed an experiment so quietly elegant it ended an argument that had occupied physics for fifteen years. The Franck-Hertz experiment fired electrons at mercury vapour and measured how they lost energy. The losses came in discrete jumps, not a continuous spectrum—proof, in a single tabletop apparatus, that the energy levels of atoms were quantised. Niels Bohr had predicted this from theory in 1913. Franck and Hertz had now shown it empirically. The 1925 Nobel Prize in Physics went to them jointly.

The war that followed interrupted everything. Hertz served in the German military from 1914, and in 1915 he was assigned to Fritz Haber's unit—the chemistry team that introduced poisonous chlorine gas as a weapon at Ypres. He was seriously wounded that same year. Haber's wife, the chemist Clara Immerwahr, killed herself shortly after the first chlorine attacks, an action many at the time and since have connected to her opposition to her husband's work. Hertz returned to Berlin in 1917 as a Privatdozent. By 1920 he had moved to industry as a research physicist at the Philips Incandescent Lamp Factory in Eindhoven. In 1925, the year of his Nobel, he was appointed Director of the Physics Institute at the University of Halle. In 1928 he took the same role at the Technische Hochschule Berlin and began developing isotope separation by gaseous diffusion.

Hertz had been an officer in the First World War, which gave him initial protection under the 1933 Law for the Restoration of the Professional Civil Service. The protection did not hold. By the end of 1934 he was forced to resign from the Technische Hochschule Berlin, classified as a 'second degree part-Jew' because his grandfather Gustav Ferdinand Hertz had been Jewish as a child before the family converted to Lutheranism in 1834. The classification was a matter of bureaucracy more than personal identity—the family had been Christian for three generations—but the racial laws did not distinguish. Hertz continued research on gas discharges, atomic physics, and ultrasound through 1944, increasingly under the auspices of industry rather than the universities that had cast him out. In April 1944 he became Director of Research Laboratory II at Siemens. He held that post until the war's end.

After the four pact members reached the Soviet Union, Hertz was made head of Institute G at Agudseri, about 10 kilometres south-east of Sukhumi on the Black Sea coast. His brief covered five projects: separation of isotopes by diffusion in inert gases (Hertz himself led this), development of a condensation pump (under Justus Mühlenpfordt), design of a uranium isotope mass spectrometer (Werner Schütze), development of frameless ceramic diffusion partitions (Reinhold Reichmann), and theoretical work on diffusion cascade stability (Heinz Barwich, who had been Hertz's deputy at Siemens). Manfred von Ardenne ran Institute A in parallel. By the late 1940s, nearly 300 Germans worked at the Sukhumi institutes. Hertz moved to Moscow in 1950. In 1951 he received the Stalin Prize, second class, shared with Barwich, and the Max Planck Medal jointly with James Franck.

Hertz returned to East Germany in 1954 to direct the Physics Institute at the Karl Marx University in Leipzig—the institution now known again as the University of Leipzig. He retired in 1961 and chaired the Physical Society of East Germany from 1955 to 1967. His marriages produced two sons, Carl Hellmuth Hertz and Johannes, both of whom became physicists in their own right. Carl is associated with medical ultrasound imaging. Hertz died in Berlin on 30 October 1975 at the age of 88. He had lived through three German political systems, worked for two superpowers, helped open the door to quantum mechanics with one elegant experiment, and helped close the door on a German atomic monopoly by walking through it himself. The cousin he barely mentioned, biologist Mathilde Hertz, had been forced from her own academic position in 1933 and emigrated to England, where she stopped publishing in her field.