The Grey family acquired Howick in 1319, in the reign of Edward II. For nearly five hundred years they lived in a fortified tower house on the site - one survey from 1715 calls it "a most magnificent freestone edifice in a square figure, flat roofed and embattled," with "a handsome court and gateway on the front." That building was demolished in 1780. Two years later, the Newcastle architect William Newton replaced it with the Georgian hall that stands today, originally entered from the south. The house was a stage set for a family that produced naval officers, generals, governors-general of India, and one Prime Minister whose name became a household word for entirely unrelated reasons.

Charles Grey, 2nd Earl Grey (1764-1845) is one of those Prime Ministers whose name is more famous in supermarkets than in textbooks. His political legacy was the Reform Act of 1832, which abolished dozens of "rotten boroughs" - parliamentary seats with almost no voters that wealthy patrons effectively owned - and extended the franchise to the urban middle class. He also presided over the Slavery Abolition Act of 1833, which ended slavery across most of the British Empire. He commissioned the architect George Wyatt in 1809 to enlarge Howick, shifting the entrance to the north side, filling out the front hall, and adding the first of the south terraces. The house we see today owes its proportions to Grey's reforming taste in architecture as well as in politics.

The story Howick tells is this: a Chinese mandarin, grateful for the Greys' kindness, sent a custom-blended black tea infused with bergamot to balance the lime-heavy water from the Howick well. Lady Grey served it to her London guests, who wanted to know where they could buy it; she sent them to Twinings, who began selling it commercially and never stopped. The historical evidence for that exact origin is thin - bergamot-scented tea was being sold under various Earl Grey labels by the late nineteenth century - but the Grey family genuinely championed the blend, and the trademark association has held for nearly two hundred years. The 6th Earl Grey, when challenged on the authenticity, simply replied that his family had always called it that.

In 1926 a fire gutted the interior of the main house and consumed the contents of the top two floors. The reconstruction in 1928 was led by Sir Herbert Baker - the architect who designed New Delhi's secretariat buildings alongside Edwin Lutyens and whose work also shapes the Bank of England - working with George Reavell. They reduced the house's footprint, opening a well in the middle and adding a portico above the front hall and a rotunda linking the front and back on the ground floor. The Grey family moved out of the main house after the 5th Earl Grey died in 1963. His grandson, Charles Baring, the 2nd Baron Howick of Glendale, converted the west wing into a family home in 1973. The hall's gardens and arboretum are now open to visitors, and the tearoom serves, of course, Earl Grey.