The site of St John's was originally occupied by the Hospital of St John the Evangelist, founded around 1200. By the early 16th century it was dilapidated and underfunded. Lady Margaret Beaufort — mother of Henry VII, patron of Christ's College, and one of the most powerful women in late medieval England — chose the hospital site at the suggestion of John Fisher, her chaplain and Bishop of Rochester. She died in 1509, before the college opened; it was Fisher who saw the foundation completed in 1511 under royal charter. Henry VIII later had Fisher executed for opposing his break with Rome. Fisher had given St John's its start. Henry founded Trinity as a rival. The hatred between the two colleges — and, as legend has it, the reason Trinity's older courts have no J staircases — traces back to that original enmity.

St John's now has eleven courts, spread on both sides of the Cam. First Court, converted from the original hospital buildings between 1511 and 1520, contains the Great Gate adorned with the Red Rose of Lancaster and mythical heraldic creatures called yales. Second Court, built from 1598 to 1602, has been described as "the finest Tudor court in England"; its Long Gallery, 148 feet long before it was subdivided, hosted the signing of the treaty that established the marriage of Charles I to Henrietta Maria. In the 1940s, parts of the D-Day landings were planned in that same room. The dining hall was extended by Sir George Gilbert Scott in 1863; its hammerbeam roof is painted black and gold. In 1564, Queen Elizabeth I rode into the hall on horseback during a state visit to Cambridge. The hall still serves formal dinners six evenings a week.

William Wordsworth studied at St John's from 1787 to 1791. He was not, by his own account, a diligent student — he found the formal curriculum uninspiring — but the Cam, the courts, and the flat Cambridgeshire landscape entered his poetry. Two abolitionists who led the movement to end slavery in the British Empire also studied here: William Wilberforce, whose parliamentary campaigns ultimately succeeded with the Slavery Abolition Act of 1833, and Thomas Clarkson, who spent years gathering evidence of the slave trade's cruelties to support the case for abolition. The college that housed these men also produced seven prime ministers — including Manmohan Singh, Prime Minister of India from 2004 to 2014 — and Nobel laureates including Paul Dirac (Physics, 1933), Frederick Sanger (Chemistry, twice: 1958 and 1980), and Abdus Salam (Physics, 1979).

St John's and Trinity have competed for centuries, across academics, sport, and legend. The rivalry is cordial in practice but ancient in origin. Two small muzzle-loading cannons on Trinity's bowling green point toward St John's, though the alignment may be coincidental. The eagle on top of St John's New Court is said to have been sculpted so that it deliberately avoids looking at Trinity. New Court's central cupola has four blank clock faces — the subject of various legends, the most popular claiming that St John's and Trinity raced to build the tallest chiming clock tower in Cambridge, with Trinity winning. The clock faces were never added. The truth is more mundane: the architect probably feared asymmetry, and the money ran out. But the story of the race has survived two centuries. There is also the claim — legally complicated, probably apocryphal — that Fellows of St John's are among the only people outside the royal family permitted to eat unmarked mute swans.

New Court, built between 1826 and 1831 in the neo-Gothic style, is nicknamed "the Wedding Cake" for its tiered, pinnacled, battlemented silhouette. It was the first major building the college constructed on the west side of the Cam, and its prominent position when glimpsed from the river — rising above the Bridge of Sighs, dramatic against the Cambridge sky — is among the most recognized views in the city. The Choir of St John's College has sung daily services in the College Chapel since the 1670s; it has nearly 100 commercial recordings dating to the 1950s, with recent contracts with Hyperion and Chandos Records. In 2021, the choir became the first at an Oxford or Cambridge college to combine male and female voices among both adults and children. The choir, like the college, has been steadily expanding what it includes.