The building was raised in 1631 by Samuel Fortrey, a wealthy London merchant of Flemish heritage, who imported the Dutch architectural fashion of the day along with his bricks. The shaped gables, the elegant proportions, the warm brick - all signal a Holland-by-way-of-the-Thames sensibility that was briefly stylish in early Stuart England. The crown leased the house in the 1720s for the royal children, and over the next century various Hanoverian princes and princesses used it as a country lodging. It was always a secondary residence, never grand, more like a substantial private home that happened to be inhabited by royalty.

Kew is where George III spent some of his darkest hours. Beginning in 1788, his recurring illness - now generally understood to have been a form of porphyria or perhaps bipolar disorder - meant prolonged stays at Kew under the care of physicians whose treatments could be cruel. By his final years he was almost entirely withdrawn from public life. Queen Charlotte and her unmarried daughters kept watch here as the king's world contracted to a few rooms. Charlotte herself died at Kew in November 1818, in a chair that visitors can still see in its place. The house holds her presence the way old wood holds varnish - quietly, durably.

On 11 July 1818 the future William IV married Adelaide of Saxe-Meiningen at Kew Palace in a joint ceremony alongside his brother Edward, Duke of Kent, who married Victoria of Saxe-Coburg-Saalfeld. Their daughter, born the following year, would become Queen Victoria. George IV, who detested the place, planned to demolish the Dutch House outright, but his plans never quite came to fruition. William IV, possibly nostalgic about his own wedding here, commissioned drawings for a west-wing extension that also never happened. He offered the house to his sister-in-law Victoria, Duchess of Kent, as a residence for herself and the young Princess Victoria; the duchess turned it down as inadequate. The house dodged each ambition that might have transformed it - and so it survives, still recognizably a Stuart merchant's home.

Queen Victoria, who had briefly sent three of her own children to summer at Kew in 1844 before settling on Osborne House on the Isle of Wight instead, finally let the Dutch House go. In 1898, to mark her Diamond Jubilee, she transferred the building - along with Queen Charlotte's Cottage in the gardens - to Kew. The palace's stables and most of the service wing had already been demolished, probably around 1881. What remained was the compact main house, opened gradually to visitors through the twentieth century, with a recreated seventeenth-century Dutch garden laid out behind in 1969 to complete the architectural illusion.

A major restoration project began in 1996. It went deeper than fresh paint and patched plaster: master weaver Ian Dale of Scotland rewove the period drapery and fabric decoration room by room, matching designs from documentary evidence. A discreet external lift shaft was added to the west wing for accessibility, placed where a Victorian privy shaft had stood before demolition. On 21 April 2006 the future King Charles III, then Prince of Wales, hosted a dinner here to mark Queen Elizabeth II's eightieth birthday. A few days later the palace reopened to the public. Historic Royal Palaces now runs the building together with the Royal Kitchens (reopened in 2012) and the Great Pagoda (reopened in 2018) - a small constellation of preserved Hanoverian Kew, hiding in plain sight among the world's most famous botanic garden.