Tantallon belonged to the Red Douglases, a branch of the family that began when George Douglas, illegitimate son of the 1st Earl of Douglas, inherited his mother's Earldom of Angus in 1389. The Black Douglases ran the main line; the Reds ran Tantallon. In 1397 George married Princess Mary Stewart, daughter of Robert III, knitting his cabal to the royal house, and Tantallon began collecting royal hostages the way other castles collected tapestries. Isabella, Countess of Lennox, was warded here from 1425 to 1433. Alexander, Lord of the Isles, served his exile here in 1429. The Red Douglases survived for a reason. They were dangerous neighbours and dangerous in-laws, and the king's enemies kept ending up behind their walls.

Archibald Douglas, 6th Earl of Angus, married James IV's widow Margaret Tudor in 1514 and tried to use his stepson, the boy-king James V, as a puppet. The conspiracy unravelled badly. When the sixteen-year-old James V escaped Angus's grip in 1528, he declared his stepfather attainted and turned the artillery train of Scotland against Tantallon. James borrowed the guns of Dunbar Castle from its French garrison, leaving three hostages in pledge, and bombarded the headland for twenty days. The chronicler Pitscottie names the cannon: Thrawinmouth, Mow and her marrow, two great botcards, two moyanes, double falcons and quarter falcons. None of it broke through. The outer ditch kept the guns at standoff range, and only when Angus's supplies ran out did Simon Penango surrender. The king's masons then strengthened what they could not destroy.

Tantallon outlived the Tudors and the Stewarts as a working fortress but not the religious wars. The Douglases stayed Catholic after the Reformation, and in 1639, while William Douglas, newly created Marquess of Douglas, was in Edinburgh during the First Bishops' War, the Presbyterian Covenanters captured his castle. They left it in ruins. Nobody repaired it. Cromwell's army gave it another battering in 1651. By 1699 the 2nd Marquess of Douglas, drowning in gambling debts inherited from his father, sold the wreck to Sir Hew Dalrymple, Lord North Berwick, who let it decay and quarried out a certain amount of stone. Walter Scott wrote it into Marmion in 1808 and turned the ruin into a Romantic icon.

The Douglas Tower at the north-west corner has half-collapsed into the sea, but the curtain wall still stands its full height. Inside the main gatehouse the entrance passage shows its layered defences: drawbridge, three pairs of doors, portcullis, machicolations punched into the ceiling for dropping missiles. The 17th-century lectern doocot sits alone in the outer court. Beyond the ditch you can trace the ravelin, a triangular earthwork added in the 1600s when artillery had finally rendered medieval curtain walls obsolete. The headland is in the care of Historic Environment Scotland. The film crew of Jonathan Glazer's Under the Skin shot here in 2013, putting Scarlett Johansson on the ramparts because, as their location manager said, they needed somewhere the wind could buffet her. Tantallon does that.