The house began life as Cope Castle, built in 1605 for Sir Walter Cope, a diplomat and confidant of James I. The architect was John Thorpe; the estate stretched from what is now Holland Park Avenue almost down to Fulham Road, and was planted with exotic trees imported by John Tradescant the Younger from the family's gardens at Lambeth. James I came to visit. The Venetian ambassador cultivated Cope. After Cope died in 1614 the house passed by marriage to Henry Rich, who was created Earl of Holland and renamed the building accordingly. Rich backed the wrong side in the Civil War. He was beheaded in 1649. The house was occupied as an army headquarters and visited regularly by Oliver Cromwell, who is said to have plotted against the king on its lawns.

Through the 18th century Holland House was leased and then bought by the Fox family. Henry Fox, made Baron Holland in 1763, was a political fixer of monumental ambition; his son Charles James Fox became one of the great Whig statesmen of the Georgian age, the orator who opposed the war with America and championed parliamentary reform. After his death in 1806 the third Baron and his wife Elizabeth Vassall turned the house into the unofficial headquarters of the Whig party. The salon they ran was extraordinary. Lord Byron met Lady Caroline Lamb here, beginning the affair that would become the scandal of the Regency. Macaulay, Sheridan, Disraeli, Walter Scott, and Charles Dickens were all guests. The house even named a room after its resident scholar, John Allen, known forever after as "Holland House Allen."

Holland House had a quieter claim to glory. In 1804 Lady Holland, in Madrid, was given dahlia seeds (or roots, the records disagree) by the Spanish botanist Antonio Jose Cavanilles. She sent them back to her librarian, Mr Buonaiuti, at Holland House. He coaxed them into growing in the kitchen garden behind the house, and the dahlia, native to Mexico and unknown in Britain, finally took. They flower every summer in English gardens now, by the million. The Royal Horticultural Society held its flower shows on the lawns of Holland House through the early 20th century, when the estate still had the largest private grounds of any London house, including Buckingham Palace.

On a summer evening in 1939, King George VI and Queen Elizabeth attended a debutante ball at Holland House for Rosalind Cubitt, the future grandmother of Camilla, Queen Consort. It was the last great ball the house would host. The Blitz began on 7 September 1940. Three weeks later the incendiaries fell. The 6th Earl of Ilchester sold the ruin and 52 acres to London County Council in 1952 for £250,000. It became, by stages, the property of the Royal Borough of Kensington and Chelsea, who still maintain it. The east wing was converted into a youth hostel that operated until recently. The roofless south face of the old building has been deliberately preserved exactly as the bombs left it, a row of mullioned windows opening on the sky.

Most summer evenings between June and August, Opera Holland Park sets up a temporary canopy in front of the ruined south facade and stages new productions of Verdi, Puccini, and the rarer works of the Italian repertoire. The Orangery, just behind, is an exhibition space; the former Summer Ballroom is now The Belvedere restaurant. The grounds beyond hold a cricket pitch, six tennis courts, and the wilder Kyoto Garden, a Japanese garden given to London in 1991 to mark a hundred years of the Japan-Britain friendship. Walking down Holland Walk on a quiet morning, with the peacocks calling and the long brick walls of the old kitchen garden flanking the path, it is possible to forget for a moment that this was once the political centre of the Whig opposition. Then the ruins reappear through the trees, perfectly preserved in their incompleteness, and the memory comes back.