Construction on the building began around 1470, half a century before Knox was born. It already had a fine wooden gallery and a hand-painted ceiling when it passed to Walter Reidpath, then to his grandson John Arres, then in 1556 to Arres's daughter Mariota. Mariota married James Mosman, a goldsmith. Her father-in-law, John Mosman, had refashioned the crown of Scotland for James V. This was a household at the centre of the Scottish royal court, working with the materials of monarchy itself: gold, silver, the regalia worn by kings. The house they lived in was substantial by Edinburgh standards, jutting out over the High Street with its timber-framed upper floor, its painted beams glowing in candlelight when the burgesses entertained.

In 1567, James Mosman was working on a piece of jewellery for Mary, Queen of Scots when she was imprisoned at Lochleven Castle and forced to abdicate. He made a choice that would kill him: he stayed loyal. Mary's supporters were holding Edinburgh Castle during what historians call the Lang Siege. Mosman went into the castle with another goldsmith, James Cockie, and they minted coins for Mary's faction. They also valued her jewels, still locked in the castle vaults, so the gems could be used as collateral for loans to keep the resistance funded. When the castle finally surrendered in August 1573, Mosman was charged with counterfeiting. He was hanged, drawn, and quartered. His head was set on a spike. The house on the Canongate was confiscated by James VI, the boy king Mary had birthed, and given to James Carmichael.

Knox did stay here briefly during the siege of Edinburgh Castle, and the visitor's pamphlet says it is believed he died here. The popular tradition of calling it John Knox's House was first recorded around 1800. Victorian writers like Robert Chambers and Sir Daniel Wilson repeated it. The Free Church of Scotland bought the building after the Disruption of 1843, which strengthened the association. When the Town Council voted in 1849 to demolish the house, the pioneering urban conservationist Lord Cockburn fought to save it. He succeeded. The Knox connection was probably what made saving it possible. Sometimes a famous name attached to a building is the only thing that keeps it standing long enough for the truth to catch up. Today the house is administered by the adjacent Scottish Storytelling Centre, owned by the Church of Scotland Knox helped found.

Just outside the door stands the Fountain Well, also called the Netherbow or Canmore. Built in 1675 by the King's Master Mason Robert Mylne to a design by Sir William Bruce, it is one of the last surviving public cisterns that once supplied Edinburgh with drinking water. Rebuilt several times, most recently in 1997, the wellhead is a reminder that this stretch of the Royal Mile was once a working neighbourhood, not a tourist promenade. The carvings inside the house itself were discovered behind later panelling in 1849 and restored by Alexander Handyside Ritchie. The building was restored again in 1984. The house has now stood for over 550 years, longer than the Reformation, longer than Scotland's union with England, longer than the city around it has been a capital.