Cawood was probably first fortified by the Saxon King Aethelstan in the early tenth century. By the twelfth it had become an archiepiscopal residence - one of the country houses where the Archbishops of York lived and worked and entertained, ten miles south of York Minster on the River Ouse. The castle proper is first mentioned in 1181. William de Greenfield built the west end in 1306. Between 1374 and 1388 it was converted into a true quadrangular castle - four ranges around a central courtyard, the architectural fashion of the late fourteenth century. King John had hunted here a hundred years before, in nearby Bishop's Wood. The grand gatehouse that still stands was built by Archbishop John Kemp in the 1430s, using stone from the same Huddlestone quarry near Tadcaster that supplied York Minster.

George Neville became Archbishop of York in 1465. He was the brother of Richard Neville, Earl of Warwick - the Kingmaker, the most powerful nobleman in England, the man who put Edward IV on the throne and would later switch sides and try to take it back. Warwick organised his brother's installation feast at Cawood and decided to make a political point. Edward IV's coronation had been lavish; Warwick's feast at Cawood was larger. The guest list included the Duke of Gloucester - the future Richard III - and the cream of the northern aristocracy. The records of what was eaten survive: those 104 oxen and 400 swans and 25,000 gallons of wine, served over several days to a thousand guests. Whatever Warwick was demonstrating about Neville power, he was demonstrating it with a knife and fork.

Sixty-five years later the castle housed another Archbishop facing political ruin. Cardinal Thomas Wolsey - once the most powerful man in England under Henry VIII, brought down by his failure to deliver the King's divorce - came to Cawood in 1530 to take up the see of York, the only office he had left. He was popular with the villagers. He fixed years of neglect and the records suggest he was kinder to ordinary people in his disgrace than he had been in his power. But before he could be enthroned in York Minster, the Earl of Northumberland arrived at Cawood with a warrant for high treason. Wolsey was arrested, fell ill on the road south, and died at Leicester. Mother Shipton, the Yorkshire prophetess, had reportedly foretold it - that he would see York Minster but never sit in its throne. He saw it from Cawood, ten miles away.

The English Civil War finished Cawood. Initially held by the Royalists when fighting broke out in 1642, the castle was taken by Parliament, retaken briefly by the Earl of Newcastle in 1644, then recaptured by Lord Fairfax shortly after. The Parliamentarians used it as a prisoner of war camp. When peace came, the castle was "slighted" - the standard Parliamentary policy of partially demolishing fortifications to prevent their future use. The villagers helped finish the job, hauling away cut stones to build their own houses. The cellar was filled in with rubble. Only the gatehouse and the banqueting hall, useful structures with no defensive value, were left standing. The fish-ponds silted up. The land turned to pasture.

The gatehouse served as a courthouse until the 1930s, then as an officers' mess and Home Guard building during the Second World War. The Landmark Trust took it on in the late twentieth century and restored the upper floor as a self-catering holiday let - one of the strangest places to spend a long weekend in England, in a fifteenth-century Archbishop's gatehouse where Wolsey lived out his last weeks. The banqueting hall next door has been stabilised but is not habitable. The castle garth - the medieval enclosure - was bought by the parish council in the 1980s and kept as open space. At its centre is a Victorian skating pond, now home to a colony of great crested newts. The medieval fish-ponds are still traceable in the grass. Walk it on a quiet afternoon and you can hear the swans Warwick's cooks would have admired, still drifting on the Ouse.