The Norman knight Ilbert de Lacy received this rock from William the Conqueror around 1070, payment for his service during the Conquest. He built first in timber, then in stone, on a natural outcrop above what would become the town. The Domesday Survey of 1086 records "Ilbert's Castle" - almost certainly Pontefract. The de Lacys held it for generations, lost it briefly to Henry I when Robert de Lacy chose the wrong side in a royal succession quarrel, regained it, and watched the castle grow into one of the great fortresses of the north. Its donjon was a quatrefoil keep, a four-lobed design shared with only Clifford's Tower in York and the Chateau d'Etampes in France. Stranger still, the castle had a torre albarrana, a detached defensive tower linked by bridge - a Spanish military feature almost unknown outside the Iberian Peninsula. The Swillington Tower, as it was called, gave Pontefract's defenders enfilading fire along the north wall.

When the English Civil War broke out in 1642, Pontefract was Royalist and stubbornly so. Parliament's troops first arrived in December 1644 and dug in for the long work of starving the garrison out. They mined the walls, brought up artillery, and brought down the Piper Tower. In March 1645 Marmaduke Langdale, 1st Baron Langdale of Holme, rode in with Royalist reinforcements and the Parliamentarian army withdrew. The second siege began almost immediately. This time the garrison held until July, when news of Charles I's defeat at Naseby reached them and they surrendered. Parliament held the castle through 1648 - until a small party of Royalists slipped inside, retook it from within, and made Pontefract a base for raids against the surrounding country. Oliver Cromwell himself rode north to settle the matter. The final siege began in November 1648. Charles I was executed in London in January 1649. Still the garrison held.

Colonel Morrice eventually surrendered to Major General John Lambert on 24 March 1649. The townspeople of Pontefract had had enough. Three sieges had impoverished and depopulated their town. They petitioned the grand jury at York, and Lambert backed them. On 27 March Parliament gave the order: Pontefract Castle should be "totally demolished & levelled to the ground," and its materials sold off. The work of slighting was thorough. What had been one of the most impressive castles in Yorkshire became a heap of rubble and a few ragged walls. Pontefract cakes - the little black liquorice discs the town has made since the seventeenth century - kept the castle's silhouette on their seal long after the towers themselves were gone.

What survives now is mostly outline. The earthworks of inner and outer baileys. A twelfth-century section of curtain wall. The postern gate of the Piper Tower, with the foundations of its chapel. The mound where the quatrefoil donjon stood. The bases of the Great Gate's fourteenth-century semi-circular towers. Visitors can still descend into the eleventh-century cellars carved into the rock, which the Civil War garrisons used for powder and shot. In 2019, on Yorkshire Day, Wakefield Council and Historic England completed a long restoration that removed Pontefract from the Heritage At Risk register. The DigVentures excavations of 2019-20 in the drawbridge pit turned up musket balls from the sieges and mason's marks from the medieval builders, fingerprints across centuries on the same handful of stones.