Hugh d'Avranches, the second Earl of Chester, built the original castle in 1070, possibly on the site of an earlier Saxon fortification. It would have been a standard motte-and-bailey affair with a wooden tower, the quick-and-dirty frontier architecture that the Normans deployed across conquered England. In the twelfth century the wooden tower was replaced by the square stone Flag Tower, and the stone gateway to the inner bailey was built. That gateway is now known as the Agricola Tower, and on its first floor sits the chapel of St Mary de Castro, whose ceiling preserves frescoes from the early thirteenth century depicting the Visitation and miracles of the Virgin Mary. These paintings were hidden for centuries, revealed only during conservation work in the 1990s.

The crypt of the Agricola Tower held prisoners whose names echo through medieval history. Richard II was held here before his deposition. Eleanor Cobham, wife of Humphrey, Duke of Gloucester, was imprisoned within these walls after her conviction for witchcraft. Andrew de Moray, the Scottish hero of the Battle of Stirling Bridge, was confined in the same dark stone rooms. During the Wars of the Roses, the Yorkist John Neville, 1st Marquess of Montagu, was captured by Lancastrians after the Battle of Blore Heath in 1459 and imprisoned at Chester until the Yorkist victory at Northampton freed him the following year. Outside the outer bailey gate, an area known as the Gloverstone served as the handover point where condemned prisoners were transferred to city authorities for execution.

Chester Castle endured the English Civil War alongside the rest of the city, besieged from September 1645 until February 1646. In 1867, the castle's military significance took an unexpected turn when Irish Fenian Michael Davitt led a group of Irish Republican Brotherhood men from Haslingden on an abortive raid for arms. By then, the castle had already begun its transformation into something more dignified than a barracks. Thomas Harrison's neoclassical redesign created the Shire Hall, now a Crown Court, whose facade stretches nineteen bays wide with a projecting Doric portico at its center. The former barracks became the Cheshire Military Museum, and the old armoury became an officers' mess. All three buildings are listed Grade I.

Today Chester Castle exists in two registers simultaneously. The Agricola Tower is a scheduled monument built in sandstone ashlar, its blocked gateway and projecting stair turret speaking of an age when defense was the primary concern. The chapel of St Mary de Castro remains consecrated as the regimental chapel of the Cheshire Regiment, its thirteenth-century frescoes now carefully preserved under controlled conditions. A few steps away, Harrison's neoclassical buildings express a completely different set of values: civic authority, judicial order, the rational architecture of the Enlightenment. In the courtyard, a statue of Queen Victoria dated 1903 surveys the scene. The inner bailey is managed by Cheshire West and Chester Council on behalf of English Heritage, a partnership between local government and national heritage that mirrors the castle's own history of layered authority.