Clitheroe's Norman keep is the second smallest stone-built keep surviving in England. It sits on top of Castle Hill - a 39-metre outcrop of pale grey limestone, identified geologically as a Waulsortian mudmound, the southwesternmost of a chain of fossil-rich reef-knolls in the Bowland Sub-basin. The hill is rich in crinoid ossicles, gastropods, and brachiopods deposited 340 million years ago when this was a tropical sea floor. On top of all that biology, sometime in the late 12th century, Robert de Lacy raised a square stone tower with walls 10 feet thick. The interior was so cramped that the great hall and other essential buildings had to stand elsewhere in the bailey - probably where the museum education suite is today.

Clitheroe was the caput - the head - of the Honour of Clitheroe, a vast feudal estate sprawling along the western Pennines from the Ribble to the moors above Manchester. After the Norman Conquest the lands passed to Roger de Poitou, then to Robert de Lacy of Pontefract sometime during the reign of William Rufus. The castle stood at the centre of this lordship's administration for seven hundred years. A Scottish force harried the area in 1138 and won the Battle of Clitheroe somewhere nearby during the Anarchy. When Henry de Lacy died in 1311, the honour passed by marriage to Thomas, Earl of Lancaster, and when Thomas was beheaded in 1322 it escheated to the Crown. From 1351 it formed part of the Duchy of Lancaster, and the Hundred Court still sat in the castle's great hall, the steward acting as judge.

By 1602 a survey called the castle "very ruinous" - parts of the buildings were collapsing. In 1644 Prince Rupert briefly garrisoned it during the Civil War, only to abandon it after the Royalist defeat at Marston Moor. Parliament ordered it slighted in 1649. In 1660 Charles II gave the castle and its honour to George Monck, 1st Duke of Albemarle, as reward for engineering the king's restoration. Ownership eventually passed to the Dukes of Buccleuch and then to Lord Montagu of Beaulieu. In November 1920, public subscription raised £15,000 to buy the site - £9,500 for the purchase, the rest to lay out the grounds as a war memorial to the 260 Clitheroe men who died in the First World War. The stone soldier in the park, head bowed and arms reversed, was sculpted by Louis Frederick Roslyn.

The buildings on the castle site house Clitheroe Castle Museum, which reopened after a £3.5 million refurbishment in 2009. Inside the Steward's House are 5,000 social history objects, a geology collection with four type and figured specimens reflecting the limestone reef beneath the castle, archaeology recovered from on-site digs, and natural history from the surrounding Bowland fells. In the rose garden south of the keep stands a curious centrepiece: a stone turret from the Houses of Parliament, presented to Clitheroe in 1937 by its MP Sir William Brass to mark the coronation of George VI, salvaged from the mid-1800s rebuilding of the Palace of Westminster. Climb to the keep on a clear day and you see the same view Robert de Lacy saw - the Ribble Valley below, the moors of Bowland to the north, and the long ridge of Pendle Hill to the east, where the Devil reputedly stood and missed his throw.