The first Chatsworth was built by Bess of Hardwick, one of the most formidable women of the Tudor age. Beginning in 1553, Bess drained the riverside site by digging a series of reservoirs that doubled as fish ponds, and constructed a substantial Elizabethan house. She lived there with her fourth husband, George Talbot, sixth Earl of Shrewsbury, who in 1568 was entrusted with the custody of Mary, Queen of Scots. Mary was brought to Chatsworth several times from 1570 onward, lodging in what is still called the Queen of Scots apartment. She remained a prisoner for the rest of her life, eventually executed in 1587. Bess outlived four husbands and accumulated enormous wealth, but her relationship with Shrewsbury soured over the cost of keeping Mary, and the couple separated bitterly. Bess went on to build the even grander Hardwick Hall nearby, but Chatsworth remained in the Cavendish family.

The house visitors see today is largely the creation of the first Duke of Devonshire, who rebuilt Chatsworth between 1687 and 1707. William Cavendish, originally planning only to replace the south front, eventually demolished and rebuilt the entire house in the Baroque style. The west front, possibly designed by Thomas Archer, features nine bays with a central pediment supported by four columns, and its carved stonework gives it a liveliness unusual in English classical architecture. The Painted Hall, with its ceiling and walls decorated by Louis Laguerre and Antonio Verrio, became one of the most celebrated interiors in England. Successive dukes added to the collections: the second and third dukes were connoisseurs who acquired Old Master drawings, ancient coins, and Greek and Roman sculptures. Palladian furniture designed by William Kent was commissioned by the third Duke for his London residence and later moved to Chatsworth.

The sixth Duke of Devonshire, known as the 'Bachelor Duke,' transformed Chatsworth's grounds with the help of his head gardener Joseph Paxton, one of the most inventive horticultural minds of the nineteenth century. Paxton arrived at Chatsworth in 1826 at the age of twenty-three and stayed for more than thirty years. He built the Great Conservatory, a vast glass and iron structure that was the largest greenhouse in the world when completed in 1840, and designed the Emperor Fountain, whose jet reaches 290 feet and remains the tallest gravity-fed fountain in England. The Great Conservatory became the prototype for Paxton's even more ambitious Crystal Palace, built for the Great Exhibition of 1851 in London. The sixth Duke also enlarged the house with a massive north wing designed by Jeffry Wyatville, adding a ballroom, sculpture gallery, and theatre. Meanwhile, the parkland that Capability Brown had redesigned in the 1760s, sweeping away the formal gardens in favour of naturalistic landscape, continued to define the estate's character.

Death duties savaged Chatsworth twice in the twentieth century. When the eighth Duke died in 1908, over 500,000 pounds became due, compounding debts from the seventh Duke's failed industrial ventures at Barrow-in-Furness. Rare books, including volumes printed by William Caxton, were sold. When the tenth Duke died unexpectedly in 1950, inheritance tax of eighty per cent of the estate's value was assessed at roughly five million pounds. The family sold thousands of acres, dozens of farms, and significant artworks to clear the debt. Andrew Cavendish, the eleventh Duke, and his wife Deborah, the youngest Mitford sister, spent decades stabilising the estate and opening it to visitors. Deborah added the maze, kitchen garden, and cottage gardens, commissioning modern sculpture alongside the historic collections. Today Chatsworth receives over 600,000 visitors annually. The twelfth Duke, Peregrine Cavendish, continues the family's improbable balancing act: maintaining one of England's greatest houses while adapting it for a world that has moved on from the aristocratic certainties that built it.