St Briavels Castle takes its name, probably, from the obscure Welsh saint Brioc. It was built between 1075 and 1129 by the great Norman family of Walter de Gloucester and his son Miles, both sheriffs of Gloucester, as the administrative headquarters of the Forest of Dean. The Norman forest was not a wilderness; it was a royal industrial estate, expected to deliver venison, timber, charcoal, and iron to the king. St Briavels stored the iron before it was shipped out. The original castle was a motte-and-bailey: a square keep on a clay-and-stone mound, perhaps twenty metres tall when complete, ringed by a stone curtain wall whose irregular polygon shape suggests it was built atop an earlier earthwork. At one end of the domestic range, the chimney is still capped by a stone forester's horn, the symbol of the warden's authority under Forest Law.

King John loved hunting in the Dean. Each November he came down to St Briavels with his court and chased deer through the oak woods. In 1207 he entertained the Welsh lord Gruffydd ap Cadwallon here. His expenditure on the castle, around 291 pounds in a four-year stretch, was substantial; the iron-working revenues of the surrounding forest probably paid for most of it. After John's death the castle's character shifted from royal lodge to royal arsenal. Under Edward I, between 1292 and 1293, the king spent 477 pounds on the castle's most spectacular feature: a massive twin-towered gatehouse, possibly by the royal master mason James of Saint George. Two huge D-shaped towers flank a passage nearly fifteen metres long. Pevsner called it "magnificent... a very fine example of the royal masons' work of the period." Uniquely in England, the passage was defended by three sets of portcullises. Why such overkill, this far from the Welsh border in peacetime? Probably to protect the weapons and the money stored inside.

After Edward I conquered Wales in 1282, the castle's strategic purpose evaporated. The structure decayed slowly across the following centuries, passing between royal favourites and disgraced earls: held by Roger d'Amory under Edward II, by Thomas le Despenser under Richard II, by the Earl of Warwick under Henry IV, then by various Yorkists and Lancastrians during the Wars of the Roses. In 1471 William Herbert was rewarded with the castle for backing Edward IV, then executed by the rival Earl of Warwick, who took it. Warwick himself fell at Barnet that same year. Under such management the keep gradually collapsed and was demolished in the eighteenth century. The chapel was restyled in the seventeenth. By the time Georgian visitors arrived, the buildings were, in one writer's phrase, "patched and cobbled like a worn-out shoe." The Forest Law still operated; the constable still presided. The lower rooms of the gatehouse had been converted into a jail.

Conditions in the castle prison were notorious. The reformer John Howard documented them in 1775. Parliamentary commissioners after the 1831 Forest of Dean riots, led by Warren James, recorded them again. They found a cell with one window a foot wide that did not open, a privy without ventilation that emptied into a hole at the building's foot, no water available to prisoners. A graffito carved into the stone reads simply: "For I have been here a great space; And I am weary of the place." Fines of up to seven pounds, the equivalent of more than five hundred in modern money, were charged against poor miners and metal-workers prosecuted under Forest Law, which had been imposed in 1217 and barely updated since. Reform finally came after the 1831 report. In 1838 the constableship was abolished and the prison closed in 1842. The buildings became a school. In 1906 they were made habitable again, and in 1948 the Youth Hostels Association moved in. The moat was infilled in 1961 and turned into a garden. The castle is Grade I listed and a scheduled monument, owned by English Heritage. The bunk beds still book up months ahead.