The plasterwork is decoration. Underneath sits a fifteenth-century timber-framed merchant's house, with parts that may be even older. The earliest documented owner was Sir Richard of Martlesham in the fourteenth century. Through the 1500s a succession of Ipswich merchants held the building. In 1567 George Copping, a draper and fishmonger, bought it, commissioned the carved oak panelling of the ground-floor room, and added a long gallery on an upper floor, a fashionable feature of Elizabethan houses where the family could walk and admire pictures in bad weather. The bones of Copping's house are still intact. The famous facade was added a century later, when the building passed to the Sparrowe family in 1603.

Each continent gets a panel, each panel a figure or scene that reads to modern eyes as a snapshot of late-Tudor European understanding, and of its blind spots. Africa is shown as a near-naked man holding a spear. Asia is represented by a horse and a domed building suggesting a mosque. Europe appears as a woman, a horse, and a Gothic church. America is a man with a dog at his feet, a depiction that suggests the artist had read travellers' accounts of indigenous peoples but never seen any. The panels are simultaneously beautiful and a record of how Europeans saw and oversimplified the rest of the world during the age of expanding trade. Australia, undiscovered by Europeans in 1670, is simply absent. The map of the world had a fifth space waiting to be filled.

The decoration is also political. The facade bears the Royal Arms of King Charles II, restored to the throne in 1660 after the Republican Interregnum, and the words Honi soit qui mal y pense, Old French for Shame upon him who thinks evil of it, the motto of the Order of the Garter. This was a building making a statement. The Sparrowe family were loyal to the crown, and they wanted everyone walking down the Buttermarket to know it. After the years of Cromwell and Puritan austerity, the elaborate royalist plasterwork on a busy street was a kind of architectural cheer for the restored king. The Sparrowes later spun a legend that the house had sheltered Charles II himself, on the run after his defeat at the Battle of Worcester in 1651. Charles is not known to have come within a hundred miles of Ipswich. The legend is almost certainly invention. But the loyalty it expressed was real.

By 1979 the Ancient House was on the verge of collapse. The foundations had sunk unevenly, the heavy fireplaces dragging one corner faster than the rest. Woodworm honeycombed the timber. Both wet and dry rot had taken hold. Deathwatch beetle, the tiny insect whose mating call sounds like a clock and which the seventeenth century associated with imminent death, was eating through structural beams. Restoration began in 1984 and lasted years. Every part of the building was touched: foundations underpinned, infestations exterminated, floors strengthened, plasterwork patched, windows re-leaded. The work uncovered surfaces and details that had been hidden for centuries. The fireplaces, having sunk further than the foundations, had to be carefully repositioned without breaking apart the surrounding fabric. It was reconstruction at the level of micro-archaeology.

Ipswich Borough Council acquired the Ancient House in 1980. For decades the kitchenware chain Lakeland leased the ground floor as a shop. They left in 2021. The building is currently between tenants, but the attic functions as an occasional art gallery, where exhibitions are held under the same beams that once roofed a wool merchant's long gallery. The Ancient House sits where it has sat for six centuries, between the modern shopfronts of Ipswich's pedestrianised centre. Tourists look up. They mostly do not know what they are looking at, only that something extraordinary is fastened to the building, plaster figures from another world telling a story about a world that thought it knew itself. The Tudor map of the continents survives, in lime and lath, on a street in Suffolk.