The antiquarian John Leland, riding through Yorkshire around 1545, collected local stories about Wada. He was, the country people insisted, a giant who had built castles and laid roads across the moors when the world was young. He came from Halsingland in Sweden, and he was buried somewhere nearby under a pair of enormous standing stones. None of this could be verified. None of it had to be. Wada was the answer to a particular Yorkshire question: who put the old stones here, in this ridge above the Esk, that nobody can quite explain? A giant did it. A giant always does it. The ruined fragments of an early fortification on the Mulgrave ridge were absorbed into the legend, and remained the giant's castle for centuries.

Nigel Fossard built the real castle - or rather the documented one - on the same ridge. By 1088 he was a tenant-in-chief of King William, holder of the feudal barony of Mulgrave, with thirty-three and a half knights' fees to his name. The castle occupied the entire width of the ridge, approached from the west between two stone towers. The ground sloped unevenly around the walls, which had to be buttressed to keep them from bulging outward. After Fossard came the Mauleys, who held the place through seven generations of Peters - Peter I, Peter II, Peter III, all the way to Peter VIII, who died without an heir in 1415. Peter III's seal, with its diagonal black bend across a gold field, hangs from the Barons' Letter of 1301, addressed to the Pope, sealed at Lincoln by seventy-two barons.

By the time of the English Civil War, the castle had passed to Edmund Sheffield, Lord Sheffield of Butterwick, whom Charles I created Earl of Mulgrave. The Sheffield family were Royalists. They garrisoned the castle for the King. After the war, Parliament made its usual decision about Royalist strongholds: it ordered Mulgrave slighted - blown up and pulled down so it could never serve as a fortress again. The order was carried out in 1647. The Norman walls came down and the medieval keep was reduced to the rubble field of broken masonry that still lies in the woods today, slowly being swallowed by ferns and oaks. A century later, Romanticism would arrive and the ruins would become an asset rather than an embarrassment, deliberately preserved as the picturesque backdrop to the new house.

The current Mulgrave Castle - a battlemented country house in stone, not a fortress at all - was built by Lady Catherine Darnley in the early eighteenth century. In 1718, her daughter Catherine Sheffield married William Phipps, and the estate passed to the Phipps family, who still hold it. They became Barons, then Earls, then Marquesses of Normanby. A nineteenth-century Marquess hosted the deposed Maharajah Duleep Singh here in the 1850s; the maharaja loved the shooting. In 2003, the supermodel Elle Macpherson and her partner Arpad Busson took a two-year lease on the estate, considered one of the finest shooting grounds in England, with the right to occupy the castle itself during the four-month season. The current owner is Constantine Phipps, 5th Marquess of Normanby.

What is remarkable about Mulgrave is that all three versions still exist on the same square mile of ground. The giant's castle survives as a few weathered stones that may have been a wall or may have been nothing at all - the country people decided which. The Norman castle survives as a substantial ruin in the woods, two stone towers and the outline of the curtain, accessible by footpath from the village of Lythe. The Georgian house survives because the family who owns it has stayed there for more than three hundred years. Three castles, three centuries of taste in what a castle should be, three answers to the question of why you might build something so emphatic on a ridge above the North Sea.