The Temple family started in sheep. Peter Temple rented pasture at Burton Dassett in Warwickshire in 1546. His son John bought the Stowe estate from the Gifford family in 1589, having previously leased it. The Temples kept rising. Sir Thomas Temple bought a knighthood from James I in 1603 and a baronetcy in 1611. The fortune grew. In the 1680s, Sir Richard Temple, 3rd Baronet, commissioned the architect William Cleare - formerly Christopher Wren's chief joiner - to build a new house at Stowe based on the design of Coleshill. Then the family kept building. Sir John Vanbrugh added an Ionic portico in the 1720s. William Kent took over after Vanbrugh died in 1726. Capability Brown worked in the gardens from 1741 to 1751. James Gibbs designed temples. Robert Adam designed the south front in 1771, which was then modified by Thomas Pitt and finished in 1779. The result, by the end of the 18th century, was a south facade 460 feet wide - one of the longest country house frontages in Britain and one of the finest neoclassical compositions in the world.

Stowe's gardens are sometimes described as the most influential landscape garden ever made. The historian Christopher Hussey called them 'the outstanding monument to English landscape gardening.' Lord Cobham, who inherited the estate in 1697 and lived until 1749, was a member of the Kit-Cat Club along with John Vanbrugh and Joseph Addison. Their conversations about garden design shaped what he built. Charles Bridgeman laid out formal axes between 1711 and 1733. William Kent moved the garden toward irregularity and surprise from 1731 onward. Capability Brown - who started here as a head gardener in 1741, aged 25, and went on to become the most famous landscape designer in British history - softened everything into the sweeping, naturalistic style that became the English landscape garden. The principle that the garden should look natural rather than geometrical, that vistas should open into one another, that a country house should sit in pretended wilderness rather than parterres, was largely worked out in this one set of fields in north Buckinghamshire. The gardens are now owned by the National Trust and saw 213,721 visitors in 2020-21.

The Temple-Grenville family produced an extraordinary string of statesmen. Lord Cobham's nephew George Grenville became Prime Minister of Great Britain in the 1760s. Cobham's sister Hester married William Pitt the Elder, who also became Prime Minister. Their son William Pitt the Younger became Prime Minister twice, the second time during the Napoleonic Wars. William Grenville, youngest brother of the 1st Marquess of Buckingham, became Prime Minister - and it was during his administration that the slave trade was abolished in 1807. The last family Prime Minister was William Ewart Gladstone, who married into the family. By the time the 2nd Duke went bankrupt, the family had been at the center of British politics for a century and a half. Queen Victoria and Prince Albert stayed at Stowe for several days in 1845, sleeping in a custom-furnished bedroom for which the Duke spent £5,300 in entertainment and redecoration costs. The state bed used during the visit is now in the Lady Lever Art Gallery in Port Sunlight. It is nearly fifteen feet tall.

The single most spectacular room at Stowe is the Marble Saloon, immediately behind the south portico. It was probably designed by the Italian Vincenzo Valdre, built between 1775 and 1777, with the decoration completed by 1788 at a cost of £12,000. The plan is elliptical, modeled on the Pantheon in Rome. The dome rises 56 feet above a marble floor of 72 four-foot-square slabs of Carrara marble resting on a brick vault. Sixteen unfluted Roman Doric columns of red scagliola - made by Domenico Bartoli to imitate Sicilian jasper - line the walls, with capitals and bases of white marble. The frieze running around the room contains over 280 human figures and 14 animals in plaster, all in alto-relievo, depicting the Roman suovetaurilia - a sacrificial procession. The dome above is coffered with 160 hand-made plaster panels, almost every one a different shape. The room was probably the most expensive single piece of country house interior decoration in 18th-century England. It is now the first room of Stowe to have been fully restored to its pre-1848 condition.

After the 3rd Duke of Buckingham and Chandos died without male heirs in 1889, the family tried to sell Stowe for £200,000. No one wanted it. The house was rented out until 1894, then stood empty until 1901 when the Duke's daughter Lady Mary Morgan-Grenville moved back in as a widow. Her son sold the house, gardens, and part of the park in 1921 to Harry Shaw for £50,000 - the equivalent of about £3 million today. Shaw founded Stowe School, which opened in 1923 and has occupied the buildings ever since. The school survived, but the house deteriorated steadily through the twentieth century. In 1990 the National Trust took over the gardens. In 1997 the Stowe House Preservation Trust took over the building. In 2002 the World Monuments Fund placed Stowe on its List of Most Endangered Sites. Restoration has continued in phases since, paid for by a combination of public appeals, the Heritage Lottery Fund, English Heritage, and anonymous donors - including one American philanthropist who funded the entire restoration of the south portico and Marble Saloon. The house has appeared in Indiana Jones and the Last Crusade, The World Is Not Enough, Stardust, X-Men First Class, and The Crown. It has 27 separate Grade I listed structures, nearly 0.5 percent of all the Grade I listings in England and Wales.