Boyne Castle sits a mile and a half east of Portsoy, on a promontory above the steep gorge of the Burn of Boyne. The stream protects the site on three sides. On the south, the side where attackers could approach on level ground, the builders cut a dry moat nearly sixty feet wide. The position is naturally defensive in the way that medieval Scottish builders preferred their positions to be: hard to reach, easy to watch. Around 1320 the land belonged to Thomas Randolph, 1st Earl of Moray, the great nephew and lieutenant of Robert the Bruce. It passed by marriage to the Edmonstone family, and by marriage again to the Ogilvies in 1437. The Ogilvies kept it for nearly three hundred years.

Charles McKean, the architectural historian, proposed that Alexander Ogilvy of Boyne built the present castle before 1575 for his bride Mary Beaton. Beaton was one of the four Marys, the companions chosen for Mary, Queen of Scots, who had been raised together at the French royal court. Bringing such a wife home to the wet Banffshire coast required a house worth her arrival, and Alexander built her a quadrangular castle with four round corner towers each twenty-two feet in diameter. The Queen herself visited the area in September 1562, and James Ogilvie of Cardell recorded that she stayed at the Craig of Boyne on the nineteenth. The Craig of Boyne may have been a different site nearer the sea, occupied before the present castle rose. Excavations there in 1869 turned up bone sewing needles, brass pins, and a lead tag once attached to imported cloth.

James VI of Scotland slept at Boyne Castle in July 1589. He was twenty-three years old, recently king in his own right after the long regency that followed his mother's flight to England, and his ambassadors had just returned from Denmark with news that mattered. Andrew Keith Lord Dingwall, George Young, and John Skene brought word that Anne of Denmark's marriage to him was progressing, that ships and jewels were ready, that a silver coach had been built to carry her north. The king received this news in a castle whose owner had lost his inheritance only a generation before in a particularly Scottish dynastic mess. James VI eventually inherited the English throne too, becoming James I of England. Anne became his queen. The silver coach made it across the North Sea.

The legend of the child at the window is exactly the kind of story families tell when a house has stopped feeling like home. Whether or not it happened, the Ogilvies of Boyne did stop using the castle. Occupation continued until after 1723, according to the records, but by then the family had other seats and other priorities. The walls began to come down by their own weight and by the steady chipping of weather. What remains today is substantial enough to read: the line of the four corner towers, the curtain wall, the dry moat on the south, the steep drop into the burn on the other three sides. You can stand where the open window stood. You can look down into the gulf the legend describes. Whether the nurse and the child fell or not, the gulf is real, and it has waited for them for nearly four centuries.