The City of London Corporation began commissioning portraits in 1670, more than 350 years ago, originally to hang in the Guildhall itself. By the eighteenth and nineteenth centuries the collection had outgrown the great hall, expanding through gifts and bequests to include history paintings, landscapes, and Dutch and Flemish masters. The first purpose-built gallery for the collection went up beside the Guildhall in the late Victorian era, a stone building in a semi-Gothic style designed to harmonise with its medieval neighbour. For decades, it housed the Corporation's growing trove. Then came the Blitz.

On the night of 10 May 1941, the worst single raid of the London Blitz, the original gallery building was destroyed. The losses were terrible: 164 paintings, drawings, watercolours, and prints, plus twenty sculptures. Specific works simply ceased to exist. The Guildhall itself survived, scarred but standing. The art gallery beside it did not. After the war, the Corporation prioritised rebuilding civic functions and chose to leave the gallery site as a gap for decades. It was not until 1985, forty-four years after the bombs, that the Corporation finally voted to commission a new gallery. Construction took the rest of the 1980s and the early 1990s. When the building finally opened, it had been designed in a postmodern semi-Gothic style by Richard Gilbert Scott, intended to sit comfortably alongside the medieval Guildhall and to house a collection that had grown to about 4,000 items.

The new entrance hall was built around one specific painting: John Singleton Copley's enormous canvas The Defeat of the Floating Batteries at Gibraltar, September 1782, completed in 1791. It depicts the climax of the Great Siege of Gibraltar, when the British garrison destroyed the elaborate floating gun platforms the Spanish had built to crack their defences. Copley painted it on a scale designed to overwhelm; in the entrance hall, that scale is finally given the room it needs. Around it hangs a sweeping panorama of mostly British art, with strong Dutch holdings: Hendrick Avercamp's frost-fair scene Winter Landscape with a Frozen River and Figures from around 1620, Frans Hals's Boy with a Glass and a Lute from 1626, Jan van der Heyden's Cityscape with a Church and a Square from around 1669, and Jacob van Ruisdael's Panoramic View of Haarlem from 1670.

The gallery's deepest strength is its Victorian collection. Joseph Severn painted Keats Listening to a Nightingale on Hampstead Heath in 1845, decades after his friend John Keats died in his arms in Rome. William Dyce's Henry VI at Towton, from 1860, freezes the king in mid-prayer during what is still the bloodiest day in English history. James Tissot's The Last Evening and Too Early, both painted in 1873, catch awkward Victorian social moments with a Frenchman's eye for what was off about English manners. Dante Gabriel Rossetti's La Ghirlandata, finished in 1874, sets a green-clad lute player among woven flowers. John Atkinson Grimshaw's The Thames by Moonlight with Southwark Bridge from 1884 shows the gaslit London he made his name painting. William Logsdail's The Ninth of November, 1888 captures a Lord Mayor's Show on the very streets just outside the gallery, with the crowd jostling around the gilded coach in a fog of November breath.

Vivien Knight served as head of the gallery from 1983 until her death in 2009, a period spanning the gallery's planning, opening, and first decade of operation. She wrote major books on the gallery's portraits and Victorian holdings, and her work shaped what visitors see today. The gallery now mounts changing exhibitions alongside its permanent display. Admission to the gallery is free for City residents and reasonably priced for everyone else. The Roman amphitheatre in the basement is the most-visited part of the complex, drawing schoolchildren and tourists who came expecting Victorian oils and found themselves standing in a Roman arena. Above ground, fragments of London's lost art collection survive on the walls. Below, fragments of London's lost Roman city wait in the dark. Both, in the end, were unearthed by accident, and both are now part of the same building.