Shortly after the Norman invasion of England, William the Conqueror made his trusted lieutenant William fitz Osbern the Earl of Hereford, and Earl William pushed west into the Welsh Marches. He captured Monmouth and Chepstow, expanding the earldom's grip. Around the same time, three castles rose in the Monnow valley to control the route from Wales to Hereford. Possibly Earl William himself ordered them; the records are not certain. What is clear is that the original three structures were earth and timber: ringworks and motte-and-bailey constructions, the standard Norman occupation toolkit, fast to build and reasonably effective against opponents who had not yet learned how to deal with them. The earldom's lands fractured after Earl William's son Roger de Breteuil rebelled in 1075. The three castles, however, kept their strategic logic. Whoever held them, held the road.

The Welsh revolt of 1135 changed everything. The new King Stephen, facing simultaneous trouble in England and along the border, restructured the Marcher landholdings to bring key fortifications back under Crown control. Grosmont, Skenfrith, and White Castle were grouped together as a single administrative and military lordship. The Crown rebuilt parts of Skenfrith and White Castle in stone. Between 1184 and 1186 a royal official named Ralph of Grosmont spent £128, probably on giving White Castle its first stone curtain wall and small keep. The work was incremental, the way medieval fortification almost always was, but the direction was clear: the three castles, taken together, would form a defensive network that no single Welsh raid could overwhelm. They never did quite work as a coordinated unit; the records suggest they often acted independently. But the lordship, as a legal and economic structure, held together for centuries.

In 1201, King John gave the Three Castles to Hubert de Burgh, a minor landowner who had risen as John's chamberlain when John was still a prince. Hubert would become one of the most powerful royal officials in medieval England. He started his upgrades at Grosmont, rebuilding the hall block in stone. Then he was captured fighting in France. When he returned to favour, he became Henry III's justiciar and Earl of Kent. He recovered the Three Castles in 1219 and over the next decade and a half transformed them. At Skenfrith he levelled the existing structure and rebuilt it completely. At Grosmont he created secure, high-status accommodation. The historian Paul Remfry argues White Castle's main stone phases came during Hubert's tenure too, in two waves between 1229-31 and 1234-39. The result was a coordinated programme that took three border outposts and made them collectively into one of the most impressive Marcher complexes in Wales. Hubert fell from power in 1232 and was stripped of the castles. He never got them back. But the buildings he commissioned outlasted him by eight centuries.

In November 1233 King Henry III camped outside Grosmont with his army, hunting the rebellious Earl of Pembroke Richard Marshal. Marshal's night attack scattered the king's forces in confusion; Grosmont itself, garrisoned by the Crown's man Walerund Teutonicus, did not fall. In 1262 the great Welsh prince Llywelyn ap Gruffudd attacked Abergavenny, and the lord of the Three Castles ordered Skenfrith to be garrisoned "by every man, and at whatever cost." The threat passed without an attack. In 1267 Edmund Crouchback, Earl of Lancaster, received the Three Castles, and they remained in the earldom and later duchy of Lancaster until 1825. Edward I's conquest of Wales in 1282 took most of the strategic purpose out of the castles, though they continued as administrative centres. The last military action came at the very start of the 15th century, during Owain Glyndwr's revolt. Glyndwr's son Gruffudd attacked Grosmont Castle in 1405, but an English force sent by Prince Henry, the future Henry V, arrived in time to relieve the garrison. After that, the castles slipped out of war and into long, slow neglect.

By the 16th century the castles had fallen into disuse. By 1613 White Castle was "ruynous and decayed." The Lancaster duchy held title without doing much to maintain the buildings. In 1825 Henry Somerset, the 6th Duke of Beaufort, bought the estate. His descendants broke it up in 1902, selling the castles to private owners who began modest conservation. In the early 20th century each castle was placed in the care of the state, and serious repair work followed. Today all three are managed by Cadw, the Welsh heritage agency. Grosmont, Skenfrith, and White Castle are protected under UK law as Grade I and Grade II* listed buildings. A 30-kilometre footpath, the Three Castles Walk, connects them across the rolling Monmouthshire countryside. Walk it in one long day, or split it across a weekend with overnight stops in the villages. The medieval lordship is gone. The legal and administrative ties that bound the three castles for seven centuries dissolved when the duchy sold up. But the triangle remains, exactly where the Normans built it, and the path between the three points is now walked for pleasure rather than urgency.