Sir David Lindsay, Lord Edzell, was educated in Paris and Cambridge and travelled the continent before returning home to extend his family's tower house in the late sixteenth century. His father had built the original tower around 1520. David was knighted in 1581, became a Lord of Session in 1593, and joined the Privy Council in 1598. He invited two German prospectors from Nuremberg, Bernard Fechtenburg and Hans Ziegler, to search the Edzell hills for precious metals. He corresponded with his brother, Lord Menmuir, about the relationship between the planets and the metals. He read alchemical books. And in 1604 he laid out a garden that incorporated everything he had learned about Renaissance symbolism, dying in 1610 with both the garden and a planned north range unfinished, heavily in debt because of his unruly son's behaviour.

Edzell's medieval owners hosted some of the most powerful people in Scotland. Mary, Queen of Scots, arrived on 25 August 1562, on a royal progress north to subdue the rebellious George Gordon, 5th Earl of Huntly. She stayed two nights. She convened a Privy Council meeting in the great hall, attended by the nobility of Scotland. By 1573 one of the chambers was being called the Queen's Chamber. Her son James VI visited twice: on 28 June 1580, and again in August 1589. On 5 August of that second visit the Earl of Erroll arrived at Edzell to submit to the King's mercy and accuse the Earl of Huntly of further treasons. The next morning news reached the castle that Henry III of France had been assassinated at Saint-Cloud by a Dominican friar, Jacques Clement.

The carvings on the east wall pediments combine the symbols of thistle, fleur-de-lis, shamrock and rose, celebrating the Union of the Crowns of the previous year, when James VI inherited the English throne in 1603. The images of the planetary gods are based on engravings made between 1528 and 1529 by Georg Pencz, a German artist who had studied under Albrecht Durer; the initials I.B. appear on the carving of Mars. The arts and virtues come from paintings by the Flemish artist Marten de Vos, distributed across Europe as engravings by Jan Sadeler and Crispijn de Passe. The image of Prudence at Edzell is the same one William Schaw, the King's Master of Works, had used in 1589 to welcome Queen Anne of Denmark to Scotland after her marriage to James VI. The pediments on the south wall are square. The west wall has no niches at all. Sir David died before he could finish.

The garden repeats the number seven and the number three in patterns that have invited interpretation for generations. The carvings come from images that were used in the art of memory, a mnemonic tradition associated with Freemasonry that flowered in Scottish culture during James VI's reign. The Danish astronomer Tycho Brahe built a symbolic garden at his observatory at Uraniborg, which James VI of Scotland visited in 1590 (in some sources erroneously called James IV), and Brahe's influence on Edzell's design has been noted. Sir David's nephew, the 1st Lord Balcarres, was famously interested in alchemy and the Rosicrucians. Historian Adam McLean has argued that the garden should be seen as an early seventeenth-century Mystery Temple, comparable to descriptions of ancient initiation chambers. The bath house and summer house in the corners furthest from the castle complete the scheme; the summer house still holds the only surviving oak panelling from the castle's interiors.

Cromwell's troops took Edzell briefly in September 1651, stationed there for a month during the Third English Civil War. The castle escaped the worst of the campaigns against the Covenanters because the Lindsays of Edzell happened to be on the right side at the right moment. The Lindsay family held it until 1715, when the last Lindsay lord, a Jacobite, sold to the equally Jacobite 4th Earl of Panmure for 192,502 Scots pounds, the equivalent of 16,042 sterling. Panmure forfeited his lands after the rising failed. The York Buildings Company, a London waterworks firm that had branched into asset-stripping forfeited Scottish estates, bought it from the Crown and stripped it. The Earls of Dalhousie eventually acquired it, appointed a caretaker in the 1870s, and put up a cottage for him in 1901 that now serves as the visitor centre. In 1932 the walled garden passed into state care. The castle followed in 1935. The garden was replanted in the 1930s and is now run by Historic Environment Scotland, open all year.