Robert de Vieuxpont founded the castle in the early 13th century, building a stone keep within an enclosure protected by earthen banks and wooden palisades. The Vieuxponts were powerful landowners in the northwest, holding Appleby and Brough castles as well. When the Wars of Scottish Independence erupted in 1296, Brougham became an important military base. Robert Clifford, 1st Baron de Clifford, who acquired the castle as heir to his mother Isabel de Vipont -- Roger de Clifford having married Isabel de Vipont in 1269 -- began transforming the wooden outer defenses into stone walls and added a large gatehouse. The work reflected the castle's position on the frontier between England and Scotland -- a place where defense was not theoretical but urgent. Edward I used Brougham as a stopover on his campaigns northward, staying in 1300.

By the time Lady Anne Clifford finally won her decades-long legal battle to inherit the Clifford estates in 1643, many of the family's castles -- Brougham, Appleby, Brough, Skipton, and Pendragon -- were in varying states of decay. The Civil War compounded the damage: Brougham was partially demolished by Parliamentary forces after the war. Anne threw herself into restoration with a tenacity that bordered on obsession. She repaired the keep, rebuilt the domestic quarters, and made Brougham habitable again. She divided her time among her castles, traveling between them in a kind of perpetual progress through her reclaimed inheritance. She favored Brougham above the others. It was at Brougham Castle that she died on 22 March 1676, at the age of 86.

After Lady Anne's death, the castle passed to the Earls of Thanet and subsequently to the Earls of Lonsdale. Thomas Tufton, 6th Earl of Thanet, stripped the castle of its roofing lead and timber, and Brougham fell rapidly into ruin. But the ruin attracted poets. William Wordsworth wrote 'Song at the Feast of Brougham Castle' in 1807 and featured the castle in 'The Prelude.' The association with Wordsworth and the Lake Poets cemented Brougham's place in the Romantic imagination -- a picturesque ruin set against the green fells of Westmorland, its towers and walls softened by ivy and time. The castle became a destination for the educated tourists who followed the Romantic poets through the Lake District.

Brougham Castle is now in the care of English Heritage and open to the public. The keep survives to a considerable height, and much of the gatehouse and curtain wall remains. The site's position at the meeting of the Eamont and Lowther rivers gives it a natural beauty that complements the architectural interest. Traces of the Roman fort Brocavum underlie and surround the medieval structure, making the site a palimpsest of nearly two thousand years of human activity in this river valley. The Countess Pillar, a painted stone column on the roadside nearby, was erected by Lady Anne in 1656 to mark the spot where she last parted from her mother. Each year, on the anniversary of that parting, money was distributed to the poor of the parish from the pillar. The tradition continued long after Anne's death, a small, persistent act of remembrance that captures something essential about the woman who rebuilt Brougham and chose to die there.