The Preston family began building at Craigmillar in the late 14th century, after King Robert II granted them the lands in 1374. The L-shaped tower house they raised on the rocky outcrop became the castle's core, its walls up to 3.3 meters thick, with vaulted ceilings and a guard room equipped with murder holes to welcome uninvited guests. A century later, the Prestons wrapped a courtyard wall around the tower, adding round corner towers with keyhole-shaped gun loops that served as much for decoration as defense. Above the gate, the Preston family arms sit beneath the royal arms of Scotland. By the 16th century, an outer wall enclosed gardens, a chapel, and a dovecote. The fish pond in the south gardens, shaped like the letter P for Preston, is now considered a nationally significant archaeological feature for its rarity.

Mary arrived at Craigmillar on 20 November 1566, recovering from a serious illness that had followed the birth of her son, the future James VI. The French ambassador Philibert du Croc reported she was also suffering from depression. Her marriage to Darnley had collapsed. He had been complicit in the murder of her secretary David Rizzio earlier that year, and his behavior had alienated nearly everyone at court. While Mary convalesced, her Secretary of State William Maitland of Lethington met with the earls of Bothwell, Argyll, and Huntly. They gathered first in Argyll's bedchamber, then moved to Bothwell's room, and finally raised the idea of divorce with Mary in her own chamber. What they actually signed in the Craigmillar Bond remains unknown, as the document was never found. Mary made clear she was unhappy with Darnley but was probably unaware of the conspiracy to kill him. In February 1567, Darnley was murdered at Kirk o' Field in Edinburgh.

Craigmillar's history extends well beyond the Darnley plot. English troops under the Earl of Hertford burned the castle in May 1544 during the Rough Wooing, Henry VIII's military campaign to force a marriage between his son and the young Queen Mary. Sir Simon Preston repaired the damage and served as Lord Provost of Edinburgh. In 1479, the Earl of Mar, brother of King James III, was imprisoned at Craigmillar on charges of practicing witchcraft against the king. The castle's connection to witchcraft persisted: in 1591, Agnes Sampson was accused of placing a charmed wax image in Craigmillar's dovecote, and the Preston family continued to preside over witchcraft trials into the 1660s. James VI himself visited the castle as a guest of Sir David Preston, and it was at Craigmillar that the young king decided to sail to Norway to collect his bride, Anne of Denmark, delayed by contrary winds.

In 1660 Sir John Gilmour, Lord President of the Court of Session, bought Craigmillar and modernized the west range with larger windows and fashionable interiors. But by the early 18th century, the Gilmours abandoned the castle for nearby Inch House, preferring modern comforts over medieval stone. Craigmillar fell into ruin and became a draw for Romantic artists -- J.M.W. Turner painted it, and William Byrne engraved it in 1782. Today it stands in the care of Historic Environment Scotland, its tower, courtyards, and gardens open to the public. From the top of the keep, Edinburgh sprawls to the north and west, with Arthur's Seat rising nearby. The castle remains one of the best-preserved medieval fortifications in Scotland, its concentric rings of walls a textbook of how Scottish castle architecture evolved across three centuries. That a sycamore tree in the grounds is traditionally attributed to Mary herself only deepens the sense of intimate history these stones carry.