A royal manor stood here from the 12th century, but the fortified site Edward I built in 1301 - the Peel, a timber palisade with an outer fosse - was the real beginning. Sixty men and 140 women dug the ditches in September 1302; the men were paid twopence a day, the women a penny. In 1313 an ordinary Scot named William Bunnock, who delivered hay to the English garrison, halted his wagon in the gateway so it could not be closed, and he and his seven sons leapt out from their hiding place under the hay to seize the Peel for Robert the Bruce. When the town burned in 1424, James I started rebuilding the site as a Renaissance palace - inspired, perhaps, by Sheen Palace in England, which he had seen during his long captivity there. The masons received tips called 'drinksilver'.

James IV continued the work and brought Margaret Tudor here as his bride in 1503 - the palace became her favourite Scottish home. Their son, the future James V, was born at Linlithgow in April 1512. After Flodden killed his father in 1513, the infant king was kept elsewhere, but Margaret Tudor's household at Linlithgow included the African servants Margaret and Ellen More, recorded in royal accounts; a French musician named Gilyem who built a new chapel organ in April 1513; and the diplomat Nicholas West, who came to see the baby prince and reported him 'a right fair child, and a large of his age.' James V himself completed the great courtyard fountain and the south facade. His daughter Mary was born here in December 1542, and the entire diplomatic crisis of her infancy - the threatened marriage to Henry VIII's son, the Secret Bond of the Auld Alliance supporters - played out in these rooms.

After the Union of the Crowns in 1603 the court moved to London, and Linlithgow was used only occasionally. The North Range, ruinous since 1599, finally collapsed at four in the morning on 6 September 1607. King James VI had it rebuilt between 1618 and 1622 with the carving designed by the mason William Wallace, but only Charles I ever stayed there as a reigning monarch, spending one night in 1633. The palace's swansong came in September 1745, when Bonnie Prince Charlie passed through on his march south. The fountain, tradition says, was made to flow with wine in his honour. Four months later, on the night of 31 January 1746, the Duke of Cumberland's army accidentally burned the palace by leaving lamps on straw bedding. By morning, most of the buildings were gone.

Today the palace is a roofless shell maintained by Historic Environment Scotland - the apartments are largely intact in their walls if not their interiors, and the elaborately carved hexagonal fountain still stands in the courtyard, said to be one of the finest of its kind in Britain. St Michael's Church next door, with its controversial 1964 aluminium crown steeple by Geoffrey Clarke representing Christ's crown of thorns, completes the medieval ensemble. In 2012 the French fashion house Chanel held its tenth Metiers d'Art show in the palace, with Karl Lagerfeld dressing models Stella Tennant, Cara Delevingne, and Edie Campbell in tweed and tartan. Outlander filmed scenes here. For over forty years, tours have been led by Junior Guides - pupils at Linlithgow Primary School. The palace is said to be haunted by the spectre of Mary of Guise, mother to the infant queen who was born within these walls.