What James Templer actually did in India is unclear. He was there for years. He married Mary Parlby, the sister of his business partner Thomas Parlby, in 1747. He returned to Devon with enough capital to buy a substantial estate outright in 1765 and rebuild it in the Georgian taste. The new Stover House, raised about half a mile from the ruins of Stoford Lodge, was a handsome classical pile with the standard cluster of Georgian outbuildings, set in landscaped parkland that included an artificial lake. Templer also extended his interests outward from the house: timber plantations across the estate, agricultural improvement, and the foundations of what would become a regional infrastructure empire under his eldest son. The first James Templer died in 1782 having transformed himself, in two generations, from tradesman's son to landed gentleman.

James Templer the second (1748-1813) inherited Stover and immediately put his money to work. In 1792 he built the Stover Canal, a short waterway from Ventiford to the Teign Navigation that linked his Devon estate to coastal shipping. The canal carried ball clay from the Bovey Basin pits and would later carry granite from his son's Haytor quarries. In 1786, with his two brothers, the second James rebuilt St Peter and St Paul's Church at Teigngrace as a memorial to their parents, filling it with mural monuments to the Templer family. He married Mary Buller, the daughter of the politician James Buller. The Templer family had moved, in two generations, from making money to spending it on civic monuments. They were becoming gentry of the older kind.

George Templer (1781-1843) inherited Stover in 1813 and immediately set about completing his father's transport network. The canal already linked Stover to the sea. What was missing was a way to bring his granite quarries at Haytor down to the canal. In 1820 he opened the Haytor Granite Tramway, an approximately eight-and-a-half-mile horse-drawn railway with rails carved out of granite blocks - the same material the tramway carried. The opening ceremony attracted the local nobility: Lord and Lady Clifford of Ugbrooke, Sir Thomas Dyke Acland, Sir Henry Carew, Sir Lawrence Palk. George also founded the South Devon Hunt, with the kennels at Stover. He was a builder, a sportsman, and unfortunately a poor businessman. By 1829 he had run through his capital and was forced to sell Stover House, the canal, the tramway, and most of the surrounding estate. The Duke of Somerset bought the lot.

Edward St Maur, 11th Duke of Somerset (1775-1855), took over Stover in 1829. He added a porte-cochere of Portland stone to the south face of the house, a curved double flight of balustraded stairs under a classical portico of Doric columns. He consulted James Veitch about the gardens. He housed the Hamilton art collection at Stover - paintings by Rubens, Lawrence, and Reynolds, brought into the family by his wife Lady Charlotte Hamilton, daughter of the 9th Duke of Hamilton. His son, the 12th Duke, served as First Lord of the Admiralty and Lord Lieutenant of Devon, but the next generation was complicated. The 12th Duke's heir Edward Seymour, Earl St Maur, predeceased him in 1869, leaving an illegitimate son Harold St Maur, who nonetheless inherited Stover in 1885 and became Master of the South Devon Foxhounds and laid out the Stover golf course in 1894. He moved to Kenya, died there in 1927.

At the start of the First World War, Stover House became a hospital for injured soldiers - Mrs St Maur, a former nurse, acted as Lady Superintendent - but it closed within a year. In 1932 Major Richard St Maur leased the house to Stover Girls' School, which moved from premises in Newton Abbot. The school still occupies the site, now as a coeducational institution. The house was Grade II* listed in 1986. In 1979 Devon County Council bought 114 acres of the former estate - woodland, heathland, grassland, marsh, and Stover Lake - and turned it into Stover Country Park, now a Site of Special Scientific Interest. The Templer Way, an eighteen-mile public footpath and cycleway, runs from Haytor on Dartmoor down through the route of the Templer tramway and canal to the New Quay at Teignmouth, where the granite was once loaded onto coastal sloops bound for London. Three generations of Templers shaped that line. Two centuries on, you can still walk it from end to end.