John of Gaunt, son of Edward III and one of the richest men in fourteenth-century England, visited Lincoln in 1386. By then a house had stood on the lower High Street, opposite St Mary's Guildhall, for a generation or more. Its owner was one of the Sutton brothers — vassals of Gaunt and serial mayors of Lincoln — and the building had a hall, privy chambers, a service range, and a lodging block that may have been built or refitted specifically to accommodate the duke. Above its doors were the lions of England and the arms of Gaunt, alongside the merchant's own heraldry. To live next door to a man like that, in a town built on the wool trade, was the closest a commoner could get to royalty. The Suttons could afford it. The wool merchants of late-medieval Lincoln were the city's banking class, and their townhouses were small palaces in everything but name.

Most great medieval houses fall in a single catastrophic moment — fire, war, dissolution. John of Gaunt's Palace died by inches. By the late eighteenth century the building was already half-ruinous. The Swiss draughtsman Samuel Hieronymus Grimm visited in 1784 and drew its southern range with its fine late-medieval windows, the Old Hospital abutting it on the west, the whole composition already missing its grandest features. By 1810 the hospital range was gone. By 1842 it had vanished from the city's official survey. In 1849 the remaining shell was rebuilt into what one nineteenth-century writer dismissed as an "ordinary-looking stone-built property," and that was what stood until 1964, when the last walls came down to make room for a garage extension. One of the original hall windows survived in the structure into the 1940s. A series of foundations was excavated in 2008, with thirteenth-to-fifteenth-century pottery in the spoil — the last physical trace of a building that once hosted a king's son.

The finest piece of stonework from the palace, an elaborate oriel window of late-fourteenth-century date, was rescued before the rest went. It now sits in the gatehouse of Lincoln Castle, a hundred yards up Steep Hill, where visitors pass under it without always knowing what it is. The window is the last visible architectural fragment of the merchant culture that made medieval Lincoln rich — a city whose woollen cloth, dyed with woad and weld into the famous Lincoln Green, clothed half of northern Europe. The oriel survived not because anyone valued it as the Sutton family's, but because Victorian antiquarians thought it had been John of Gaunt's. A myth, in the end, was what saved it.

The neighbourhood the palace anchored is itself a kind of palimpsest. Just to the south stood St Andrew's Hall, a Norman house from about 1150, similar in style to the Jew's House further up the hill. It was demolished in 1783; its Romanesque doorway, salvaged in pieces, was reinstalled in 1907 in what became the Lincoln Arms and is now a Bang and Olufsen shop, on the first floor in a back room, where almost no one sees it. Next door was a brick house built in 1646 for Alderman Original Peart — a name that sounds invented but was real — and later occupied by the Bromhead family. It was one of the very earliest brick buildings in Lincoln, an Artisan Mannerist oddity in a city otherwise built of pale limestone. The cumulative effect, walking the lower High Street today, is that the surface gives almost nothing away. The medieval merchant city is still here. It has just been rebuilt around itself so many times that you have to know where to look.