After the Norman Conquest of 1066, William the Conqueror made his loyal supporter William fitz Osbern the Earl of Hereford, and Earl William pushed west into the Welsh Marches, capturing Monmouth and Chepstow. To hold the route from Wales to Hereford he, or his men, threw up three timber and earthwork castles in the Monnow valley around the same time: the one originally called Llantilio Castle (later White), plus Grosmont and Skenfrith. The original Llantilio Castle was earth and timber, with three large earthworks forming an inner and outer ward and a hornwork covering the southern entrance. A mill was built nearby at Great Trerhew to grind corn for the garrison. After Earl William's son Roger de Breteuil rebelled in 1075, the earldom's lands fragmented. Then in 1135 a major Welsh revolt prompted King Stephen to gather the three castles back under direct royal control as a single lordship: the Three Castles. Between 1184 and 1186, a royal official named Ralph of Grosmont spent £128 on probably building the first stone curtain wall around the inner ward and adding a small stone keep.

In 1201, King John gave the Three Castles to Hubert de Burgh, a minor landowner who had become John's chamberlain when John was still a prince and who would rise to become one of the most powerful royal officials in England. Hubert started his improvements at Grosmont, but he was then captured fighting in France. When he returned to favour, he became Henry III's justiciar and was made Earl of Kent. He recovered the Three Castles in 1219. The next two decades transformed White Castle. The historian Paul Remfry argues the major stonework went up in two campaigns, 1229-31 and 1234-39. Walerund, who took the castles when Hubert fell from power in 1232, added a new hall, buttery, and pantry in 1244. The end result was what Remfry calls "a masterpiece of military engineering": a stone curtain wall studded with four circular towers and a great twin-towered gatehouse complete with portcullis and drawbridge, all of it ringed by a wet moat that survives today.

Walk around the inner ward and look at the arrow slits. White Castle's are unusual. The two arms of the cross-shape are offset vertically, one higher than the other. Why? Historians disagree. One theory: the offset let defenders shoot down the steep slopes around the castle more effectively. Another: it gave the man behind the loop better protection from incoming missiles. A practical test conducted in 1980 produced a discouraging answer. The offset loops actually turned out to be more vulnerable to incoming shots than the conventional design, not less. So either White Castle's builders were experimenting with a defence that did not really work, or they had a tactical reason that the 1980 test did not capture. Either way, the loops are still there, oddly modern-looking in their asymmetry, an architectural puzzle in stone. The gatehouse, surviving up to 5 metres tall, was the castle's grand statement: four storeys, two flanking circular towers, portcullis, drawbridge. It was the residence of the constable or steward. Behind it, around the inside of the curtain wall, ran the hall, chapel (partly inside one of the towers), the constable's living quarters, service buildings, and the kitchen. Only the foundations remain.

Edward I's conquest of Wales in 1282 removed White Castle's military reason for existing. The Welsh were no longer a strategic threat, and a frontier fortress 7 miles west of the English border now sat in the middle of a settled lordship. The castle continued in use as an administrative centre, a place to muster levies if needed, and a residence of declining importance. In 1267 the Three Castles had been granted to Edmund, Earl of Lancaster, and they remained in the earldom and later duchy of Lancaster's hands until 1825. Minor repairs were made: a gatehouse tower at some point, the chapel tower and gatehouse under Henry VI. But the castle slowly emptied out. By 1538 it had fallen into disuse and ruin. A 1613 description called it "ruynous and decayed." Henry Somerset, the 9th Duke of Beaufort, sold it to Sir Henry Mather Jackson in 1902, and the state took it into care in 1922. Cadw runs it today, and the castle is a Scheduled Monument and Grade I listed structure.

Visit on the right day and you can have the place largely to yourself. The wet moat catches the sky in pieces. The inner ward sits like a crown on its hill, the four round towers softened by lichen but still distinctly fortress. From the top of the gatehouse you can look south across the hornwork, the crescent-shaped outer defence that originally guarded the main approach, and out over the Llantilio Crossenny manor toward the River Monnow. To the north and east the outer ward, much larger originally than what survives, has melted into earthworks and pasture. The walk down to the moat and back is short. The Three Castles Walk, the 30-kilometre footpath that connects White, Grosmont, and Skenfrith, passes through here. The total medieval defensive scheme is best understood by walking it. White Castle was not meant to be visited alone; it was meant to be a node in a network, holding a section of the line, while Skenfrith held the river crossing and Grosmont held the road. The line is gone. The castles remain.