Charles Harrison Townsend designed the original building, and it remains a notable example of the British Modern Style - the arts and crafts cousin of Art Nouveau that flourished briefly at the turn of the twentieth century. The facade reads as something between a place of worship and a department store: terracotta ornament, twin arched portals, and an asymmetric frontage that refuses to behave like a Victorian institution. Townsend was also responsible for the Horniman Museum and the Bishopsgate Institute, and across the trio he produced some of the most distinctive non-religious public architecture in London. The gallery was funded by Canon Samuel Barnett and his wife Henrietta as part of the wider Toynbee Hall settlement movement, which planted cultural institutions in poor neighbourhoods on the belief that beauty was not a luxury but a right.

Through the twentieth century, the Whitechapel staged shows no one else in Britain would. The first UK exhibition of Mark Rothko came here in 1961 - and the way the gallery hung his canvases became the template Rothko used for every show afterward. Jackson Pollock arrived in 1958. The 1956 exhibition This is Tomorrow brought Pop Art to British audiences before the term had quite settled, with Richard Hamilton's collage Just What Is It That Makes Today's Homes So Different, So Appealing? becoming one of the founding works of the movement. The Whitechapel's New Generation show of 1964 introduced the country to John Hoyland, Bridget Riley, David Hockney, and Patrick Caulfield - artists who would shape the Swinging Sixties as the gallery quietly midwifed them.

By the early 2000s the gallery had run out of room. Next door stood the former Passmore Edwards library, named for the Cornish-born philanthropist who funded twenty-four free libraries across late-Victorian London. The library had moved to the new Whitechapel Idea Store, and in 2009 the gallery absorbed the empty building, doubling its size and tripling its exhibition space. The expansion cost about thirteen and a half million pounds and was supported by the Heritage Lottery Fund. On the roof, sculptor Rodney Graham installed a weather vane depicting himself as the sixteenth-century scholar Erasmus, riding a bicycle. The gesture - playful, scholarly, faintly absurd - captures the gallery's voice. Inside, Goshka Macuga reopened the space with a project that re-staged the 1939 Guernica show, with a full-size tapestry of the painting loaned from the United Nations art collection.

The gallery is not just a venue for the famous - it is often the first major UK survey for artists who later become very famous indeed. Sarah Lucas had her first major London solo show here in 2013. Hannah Höch, the Dada pioneer who deserved her recognition a generation earlier, finally got it here in 2014. The 2015 exhibition Imperfect Chronology was the first major British show of modern Arab art, drawing from the Barjeel Foundation's collection. The Guerrilla Girls staged a commission in 2016 alongside a retrospective of Eduardo Paolozzi. The pattern is consistent: artists from the global modern movement who have been undersung in Britain get their moment in a building whose neighbours include the East London Mosque, Brick Lane curry houses, and the relentless commerce of Whitechapel Market.

Stand outside the gallery on Whitechapel High Street and the cultural geography becomes clear. Across the road, the Aldgate end of the City. A few minutes east, Altab Ali Park, named for a Bangladeshi worker murdered in a racist attack in 1978. Brick Lane to the north. The Royal London Hospital to the east. The neighbourhood the Barnetts founded the gallery to serve has remained the most diverse and most cosmopolitan part of London, and the gallery has remained close enough to its audience that it never quite became a tourist destination. Show up on a weekday and you will find as many local students and pensioners inside as collectors and critics. That is rare, and it is the original brief made good a century and a quarter on.