The main line of the Courtenay family, the Earls of Devon, lived at Tiverton Castle in the 14th and 15th centuries. Powderham belonged to a junior cousin branch known as 'Courtenay of Powderham,' built by Sir Philip Courtenay, fifth or sixth son of Hugh Courtenay, 2nd Earl of Devon. The two branches were related but not always friendly. The Powderham Courtenays served as Justices of the Peace, sheriffs, and Members of Parliament for Devon, and were arguably the most prestigious gentry family in the county. When the senior Tiverton line went extinct in 1556, the Powderham branch inherited the claim. They became de jure Earls of Devon immediately, took their seats in the House of Lords as Viscounts in 1762, and were finally confirmed de facto Earls in 1831. They have held the title ever since. The current Earl is the 19th, Charles Courtenay.

The 'castle' in Powderham Castle is a stretch. The original building was a fortified manor house, not a true castle with keep and moat. The label 'castle' was not attached to the property until the 17th century at the earliest. It did have a curtain wall and yard on the east side, as shown in a 1745 engraving by Samuel Buck, but those were demolished during 18th-century landscaping to open up views of the Exe Estuary from the lower rooms. Most of the visibly castle-like features on the west front were added in the 19th century. The gatehouse with its battlements was built between 1845 and 1847 to a design by Charles Fowler. The medieval core remains: thickly walled, double-height, originally containing a withdrawing room, great hall, screens passage, and kitchens, all aligned north to south. Those rooms are now the ante-room, Staircase hall, Marble hall, and Victorian kitchen, but the medieval bones are still there if you know where to look.

In December 1645 the English Civil War came to Powderham. The castle was garrisoned by 300 Royalist soldiers under Sir Hugh Meredith. A Parliamentary detachment under Sir Thomas Fairfax, the senior commander of the New Model Army, tried to capture it and was at first repulsed. The castle finally fell on 25 January 1646 to Colonel Robert Hammond, the same officer who would later guard King Charles I on the Isle of Wight. The assault left Powderham badly damaged. In places it remained open to the elements until the early 18th century, when Sir William Courtenay, 2nd Baronet, finally repaired it before his death in 1735. The Marble Hall, named for its black-and-white marble floor, was completed in 1755 by partitioning the original double-height medieval great hall. Today the hall functions as a sitting room with an 18th-century fireplace, a 14-foot longcase clock by William Stumbels of Totnes made about 1745, a 17th-century Brussels tapestry, and a 1553 carved over-mantel bearing the Courtenay arms.

Upstairs at Powderham, in a case that has long delighted visitors, sits a narwhal tusk. For centuries, narwhal tusks were sold as 'unicorn horns,' said to detect poison when it touched them. Medieval and Renaissance kings paid extraordinary sums for them. The Powderham tusk is probably from such a transaction. The Dining Hall, built between 1847 and 1859 by the 10th Earl and finished by the 11th in 1860, holds linen fold panelling with dozens of ancestral heraldic shields, and a chimneypiece copied from one installed around 1485 at the Bishop's Palace in Exeter by Peter Courtenay, Bishop of Exeter and a younger son of Powderham's 15th-century master. The chimneypiece's spandrels carry dolphins, a Courtenay badge. Its supporters are Bohun swans, the heraldic bird used by the family of the wife of Hugh Courtenay, 2nd Earl of Devon, heiress of Powderham. The room is a small, dense library of family memory.

Powderham has a strange second life as an entertainment venue. The 1993 Merchant Ivory film The Remains of the Day used the staircase hall, music room, and master bedroom as locations. Rock band Status Quo performed concerts in 2003 and 2009. Westlife played their Turnaround Tour here in August 2004. BBC Radio 1's Big Weekend in May 2016 brought Ellie Goulding, Coldplay, The 1975, Craig David, Nick Jonas, and Iggy Azalea to the castle lawn. Mary Berry filmed an hour-long episode of her Country House Secrets series here in 2017. In 2019 the castle launched its own food festival. After COVID the family added a Christmas light trail. The Great Western Railway's Castle-class steam locomotive 'Powderham Castle' was named after the building in 1924; the main GWR line from Exeter to Plymouth still runs alongside the castle grounds, and passengers can see the castle from the train as they pass. Six hundred years on, the Courtenays are still finding new uses for the marshland.