Around 1100 the Norman lord Roger Bigod looked at a tight loop in the River Waveney where the river almost doubled back on itself, and he saw a defensive position that nature had already half-built. The peninsula inside the loop was protected on three sides by water. A simple wooden palisade with a single landward approach could turn the site into a serious fortress, and Roger raised one of the earliest motte-and-bailey castles in Suffolk on the spot. His son Hugh Bigod inherited both the castle and his father's instinct for political risk. During the civil war period known as the Anarchy between 1138 and 1154, Hugh played the field hard, switching sides between King Stephen and the future Henry II as advantage seemed to dictate. Henry remembered. When Henry took the throne, the Bigod castles at Bungay, Framlingham, Walton and Thetford were among the first royal worries in East Anglia, and the king set about clipping the family's wings.

Henry confiscated all four Bigod castles, then returned Framlingham and Bungay to Hugh in 1165 as a calibrated gesture, while building his own enormous polygonal keep at Orford on the coast to dominate Bigod country from outside the family's network. Hugh joined the revolt of 1173-74 against Henry in support of the king's rebellious sons. He picked the wrong side again. Royal forces besieged Bungay, and when the walls would not fall to assault the engineers began to mine. The technique was straightforward and terrifying. Tunnel under a corner of the keep, propping the excavation with timber as you went. When the tunnel reached far enough, pack it with brushwood and fuel and set the props alight. The fire would burn out the props, and the unsupported masonry above would collapse into the cavity. Hugh Bigod surrendered before the mine was fired. Henry took the castle, ordered the keep slighted so that it could not easily be re-fortified, and Framlingham was permanently confiscated. Bungay went quiet.

The Bigods recovered. The family's loyalty wavered enough in the next century to attract further trouble, but by 1294 Roger Bigod, the fifth Earl of Norfolk and the great-great-grandson of the original Roger who picked the river loop, was a major Plantagenet magnate. He probably built the massive gate towers whose ruins you see today, the twin drum towers that gave the castle its definitive late-medieval silhouette. The fifth Earl died in 1306 without legitimate heirs, and his vast holdings reverted to the crown. The castle passed through hands repeatedly over the next three centuries. In 1483 the Dukes of Norfolk reacquired it. They held it almost continuously until the twentieth century, with one short eighteenth-century interruption.

In 1766 the Dukes sold the castle to a local man named Robert Mickleborough, who saw what the medieval engineers had built and saw road metalling. Mickleborough quarried the keep and curtain walls for building stone, in what amounted to an industrial-scale removal of much of the surviving castle into the road network of late-eighteenth-century Suffolk. In the early 1790s a local solicitor named Daniel Bonhote bought what was left. Around 1800 the Dukes of Norfolk bought it back. The damage was done. The castle that survived into modern times is the gatehouse ruins, parts of the curtain wall, and the earthwork remains, all Grade I listed, standing in the river loop where Roger Bigod first chose his site nine hundred years ago. The Bigods are long gone. The Waveney is still doing what brought them here in the first place.