John de Vaux was an Anglo-Norman knight whose family came originally from Rouen and crossed to England with the Conqueror. King David I of Scotland, who in the 12th century systematically invited Anglo-Norman knights north to professionalise his kingdom, granted John de Vaux the barony of Dirleton. Around 1240 his descendants began building in stone on a natural rocky outcrop overlooking the rich agricultural land of East Lothian, guarding the coast road from Berwick to Edinburgh. The donjon, or main tower, that still stands today is a survival from that first stone castle. Its design has been linked to the Chateau de Coucy in northern France, suggesting that the de Vaux family were importing not just their own architectural taste but the cutting-edge French castle design of the era.

Peaceful times ended in 1296, when Edward I of England launched the campaign that began the Wars of Scottish Independence. Dirleton, guarding the road to Edinburgh, became a prize that changed hands repeatedly. In the summer of 1298 the formidable Antony Bek, Bishop of Durham, brought an English army to the gates. The castle held out for several months. Only after the English victory at Falkirk freed up the great siege engines could Bek reduce Dirleton. The Scots retook it sometime before 1306, when Aymer de Valence, Earl of Pembroke, marched in and captured it for the English again. Sometime before 1314 the Scots took it back one last time, then deliberately damaged it, slighting the walls to prevent its reuse by an enemy who kept coming back.

The Haliburton family rebuilt and expanded the castle through the 14th and 15th centuries, heightening the original towers, adding a great hall and tower house along the east range, and constructing a new gatehouse. In 1505 the castle passed by marriage to the Ruthvens, a family whose name appears in nearly every dark moment of late 16th-century Scottish politics. Patrick, 3rd Lord Ruthven, was one of the leaders of the gang who burst into Mary Queen of Scots' supper chamber at Holyrood in March 1566 and murdered her private secretary David Riccio in front of her. Two generations later, the Gowrie Conspiracy of August 1600 ended with the deaths of two more Ruthven brothers, John (3rd Earl of Gowrie) and Alexander, in a confused fight at Gowrie House. The official story said the brothers had tried to assassinate King James VI; the contemporary suspicion was that James had ridden to Gowrie House specifically to kill them. Whatever the truth, James seized the Ruthven lands. Dirleton was given to Thomas Erskine, who had been at Gowrie House that night, in what many called blood money.

The castle's last serious fight came in November 1650. Oliver Cromwell's army had invaded Scotland during the Third English Civil War and crushed the Scottish Royalists at the second Battle of Dunbar in September. With southern Scotland under English control, bands of Royalist moss-troopers harried English supply lines from hideouts in the country. One such band based itself at Dirleton. Cromwell sent Generals Monck and Lambert with 1,600 troops to take the castle. They brought mortars. The drawbridge and inner gate were destroyed by mortar fire on 10 November. The captain of the moss-troopers and two of his comrades were hanged from the walls. The castle, slighted again, was briefly used as a field hospital before being abandoned to slow decay. The man responsible for Cromwell's effective use of artillery here was Major Joachim Hane, a German engineer who would go on to direct the mortar fire at the siege of Stirling Castle the following August.

Between 1649 and 1650, in the same months Cromwell was preparing his Scottish invasion, almost 600 people across south and east Scotland were accused of witchcraft. Six of them were from Dirleton. The interrogations were probably conducted by the local church court under John Makghie, minister of Dirleton Kirk. Agnes Clarkson, a widow, confessed in June 1649 to having been visited first by a woman from Longniddry who had already been burned for witchcraft, then by the devil himself in the form of a black dog. The records that survive name the accused but rarely tell us how they died. Most witch convictions in 17th-century Scotland ended in strangling at the stake, followed by burning of the body. These were real women, mostly poor and middle-aged, swept up in an episode of communal hysteria that the modern Scottish government formally apologised for in 2022. Today the castle and its gardens are looked after by Historic Environment Scotland. The 215-metre herbaceous border, the Arts and Crafts garden laid out in the 1920s, the 16th-century doocot with its thousand nesting boxes, and the surviving ruins all sit together. A castle this old contains too many stories for any one mood; Dirleton offers all of them at once.