Taunton's fortifications reach back to around 710 AD, when King Ine of Wessex and Queen Aethelburg established a stronghold here. The Anglo-Saxon Chronicle records that this first fortification was destroyed twelve years later, though it may not have stood on the exact site of the present castle. An ecclesiastical minster was traditionally founded at Taunton shortly afterward by Queen Frithugyth, wife of King Aethelheard of Wessex, and the Bishops of Wessex appear to have built a manor house adjoining it. By the time of the Domesday Book in 1086, Taunton belonged to the Bishop of Winchester, who maintained a minster or Augustinian priory on the site. The transition from religious house to military fortification came between 1107 and 1129, when William Giffard, Chancellor to King Henry I, converted the bishop's hall into a castle.

Taunton Castle belonged not to kings or barons but to the Bishops of Winchester -- one of the wealthiest ecclesiastical sees in medieval England. This unusual ownership shaped the castle's character. In 1216, Bishop Peter des Roches, a supporter of King John, defended it during a barons' revolt. Later bishops added Tudor windows and heraldic displays: a tablet bearing the bishop's arms, supported by angels, was placed above a large window, with the Royal Coat of Henry VII below. The Great Hall, standing just opposite the gateway, has walls that are partly Norman but so extensively modified over centuries that the building reads as a history of English architectural fashion. Bishop Richard Foxe paid for a schoolhouse within the walls in the 1520s, the last significant medieval construction at the site.

During the English Civil War, Taunton became a Parliamentarian stronghold in Royalist territory -- a position that invited siege. The town was briefly relieved by Sir William Waller, but the Royalists closed in again. A relief column under Colonel Ralph Weldon reached Taunton on 11 May 1645, yet the combined Parliamentarian forces remained besieged within the castle and town. They held out until 14 June, when help finally arrived after troops could be spared from the Battle of Naseby, the decisive engagement that broke Royalist power. Taunton's stubborn resistance had tied down Royalist forces at a moment when King Charles could least afford the distraction. The castle's survival was less about its medieval walls than the determination of the garrison behind them.

After the Bloody Assizes cemented the castle's grim reputation, its military significance faded. By 1780, much of the building had deteriorated and was repaired in Georgian style by Sir Benjamin Hammet, a London banker and Member of Parliament for Taunton. Hammet installed new windows, rebuilt the roof, and recast details throughout the castle -- practical work that saved the structure but buried much of its medieval character under eighteenth-century plaster. During the following two centuries, the Great Hall served as a public meeting place, a courthouse, and eventually a museum. In 2011, the inner ward reopened as the Museum of Somerset after a seven-million-pound refurbishment partially funded by the Heritage Lottery Fund. The adjacent seventeenth-century Castle House, designated an Ancient Monument, required its own repairs and was placed on the Heritage at Risk Register. The castle that began as an Anglo-Saxon fortification and became a bishop's residence now tells the story of Somerset itself -- from Iron Age artifacts to the worn courtroom benches where Jeffreys sat in judgment.