The land was originally Anglo-Saxon, then Norman, then granted by Ralph de Berners in 1253 to the Augustinian Canons of St Bartholomew's Priory in Smithfield. The area was called the Canons' Burgh - hence Canonbury. In 1509 William Bolton became Prior of St Bartholomew's, and besides restoring the priory church and serving as Henry VIII's Master of the Works on the Henry VII Chapel at Westminster Abbey, he built or rebuilt the Canonbury manor house and its strange six-storey tower in the 1520s. The Tower's purpose was obscure even at the time: it had no defensive function. The view from its flat roof took in Middlesex and the City - in 1807 a visitor wrote that "the smoke of London was not then so dense" and "very few buildings intervened." Above the door of one of the surviving Elizabethan garden houses, a small carved emblem still shows a bolt piercing a tun - a punning rebus on Prior Bolton's name. Bolton died in 1532. Seven years later the Dissolution swept St Bartholomew's away, and Henry VIII handed Canonbury to his chief minister Thomas Cromwell. Cromwell held it for less than a year before being executed in 1540.

John Spencer first leased the property from Thomas Wentworth in 1570 for £21.11s.4d, then bought it outright for £2,000. "Rich Spencer" was Lord Mayor of London in 1594 and possessed one of the great private fortunes of his day; Queen Elizabeth is said to have visited him at Canonbury in 1581. Spencer took up regular occupation in 1599 as his country house, modernising the building with stuccoed ceilings, oak panelling, and elaborate carved chimney-pieces - many of which still survive in the Tower today. His layout placed three ranges of buildings around a courtyard, with the Tower at the north-west corner and a bell turret with a cupola above the east range. The walled grounds extended south toward the New River, with the two Elizabethan octagonal summer houses marking the garden corners. After the elopement and reconciliation, the fortune passed to William Compton, who became 1st Earl of Northampton in 1618. He didn't live at Canonbury much; the house was let, while Compton spent his money on Castle Ashby in Northamptonshire.

From 1616 to 1625 Canonbury House was leased to Sir Francis Bacon, then at the height of his career as Attorney General and Lord Keeper. The Tower's first-floor Spencer Room still bears decorative details - Tudor roses set within what might be garters, strapwork ornament running along the underside of the mantelpiece - that some have read as prefiguring Bacon's interest in Rosicrucian symbolism. On the second floor, the Compton Room is more elaborate again, with carved figures of Faith and Hope flanking the chimneypiece and Latin inscriptions reading Fides Via Deus Meta - "Faith is my way, God is my aim" - and Spes Certa Supra - "My sure hope is above." Tradition holds that Bacon planted the red mulberry tree that still grows in the Tower's garden, hoping to encourage English silk production. The plan was flawed: silkworms prefer white mulberries. The mulberry tree, like much else here, became a story long after the man who supposedly planted it had moved on. Oliver Goldsmith took rooms here in the 18th century while writing The Vicar of Wakefield. Washington Irving stayed too. The Tower, by its nature small and inward-looking, attracted writers who needed somewhere quiet to think.

Canonbury Tower is 66 feet high and roughly 17 feet square. The brick walls vary in thickness from 4 feet to 2 feet - thicker at the base, tapering as they rise. The central staircase, in short straight flights with quarter landings, was built around a core of timber and plaster forming a series of cupboards. The black oak balusters are mostly original. At the very top, when restorers worked on the Tower in 1907-08, they cut a new handrail newel and balusters from sound oak beams found buried in the woodwork - four centuries old, but, when sawn open, still fresh and sweet-smelling. The roof, originally pitched rather than flat, carries a copper weather vane. In the space immediately below the roof, a small room - probably once a schoolroom - bears a list of the kings and queens of England written in rough Latin and Norman French hexameters, ending at Carolus qui longo tempore ("Charles, who reigned for a long time"). On the wall is a mnemonic in Latin elegiac couplets: "Let your thoughts be on your own death, the deceits of the world, the glory of heaven, and the pains of hell." Tudor pedagogy did not believe in soft messages.

The buildings around the old courtyard survive in scattered pieces. Nos. 6-9 Canonbury Place, on the site of the east range, retain genuine 16th-century stuccoed ceilings and oak carvings - significant enough that they were listed for the quality and rarity of the plasterwork alone. From 1993 the buildings briefly housed Canonbury Academy, a conference centre that local rumour - probably wrongly - insisted was a cover for MI5 training. Since 2014 they have been the premises of North Bridge House Senior School. The Tower itself has been used since 1998 as the home of the Canonbury Masonic Research Centre and can occasionally be visited on guided tours. The two octagonal Tudor garden houses still stand: one at 4a Alwyne Villas with its brickwork exposed, one at 7 Alwyne Road rendered in cement. Charles Dickens set a short story in Canonbury in 1838, with the Tower as a setting. George Reynolds serialised a penny-dreadful novel called Canonbury House; or, the Queen's Prophecy in 1857-58. The place keeps its strange Tudor footprint in a neighbourhood that has otherwise become Georgian terraces and Victorian villas - a six-storey survival from before any of that, holding its baker's-basket legend and its mulberry tree and its inscribed warning about the pains of hell.