The abbey began in the tenth century as a Benedictine foundation for women, receiving a land grant from King Edgar in 968. Within a generation, disaster struck. Viking raiders sacked the village and burned the church in 993. But the community rebuilt in stone around the year 1000, and the abbey became renowned as a center of learning, particularly for the children of the nobility. In Norman times, Henry Blois, Bishop of Winchester and brother of King Stephen, oversaw the construction of a substantial new stone church on the Anglo-Saxon foundations. By 1240, the community had grown to over a hundred nuns. Then came the Black Death in 1348. Half the town's population of about a thousand died, and the number of nuns collapsed from over ninety to just nineteen. Seventy-two nuns perished, including Abbess Johanna. The community never exceeded twenty-six again.

When Henry VIII dissolved the monasteries in 1539, most religious houses across England were demolished for their building materials. Romsey survived because its church contained a section dedicated to St Lawrence that served as the parish church for the townspeople. This dual use, a common arrangement designed to separate monastic worship from the public's needs, saved the building from the wrecking crews. The town purchased the abbey from the Crown in 1544 for a hundred pounds, though the local magnates promptly demolished the very St Lawrence section that had justified the purchase, stripping it for lead and stone. During the English Civil War, Parliamentarian troops damaged the building further in 1643, destroying the organ. What survives today owes much to the efforts of the Reverend Edward Lyon Berthon, a nineteenth-century incumbent who championed the building's restoration.

Romsey's bells have their own story. Originally housed in a detached campanile that was demolished in 1625, the bells were moved to a wooden belfry atop the central tower. A new set of eight was cast in 1791, with three recast in 1932. A full restoration in 2007 reduced the weight of the tenor bell from 26 hundredweight to 22 by removing the crown, and the bells are now regarded as one of the finest rings of eight in the region. The abbey also maintains an active musical tradition with multiple choirs, two organs, and regular appearances on BBC Songs of Praise. The main organ, built by J. W. Walker and Sons in 1858, was rebuilt in its present position in 1888 and thoroughly restored in the 1990s.

Among the tombs in Romsey Abbey, one draws visitors above all others. Earl Mountbatten of Burma, the last Viceroy of India and Supreme Allied Commander in Southeast Asia, was buried here following a ceremonial funeral at Westminster Abbey in September 1979. Mountbatten had been assassinated on 27 August by a bomb planted by the Provisional IRA aboard his fishing boat at Mullaghmore, County Sligo. His grandson Nicholas, a local teenage boat crew member Paul Maxwell, and Doreen Lady Brabourne also died. At Mountbatten's request, his grave is aligned north-south rather than the conventional east-west, so that he faces the sea where his wife Edwina's ashes were scattered. It is a personal gesture within a building that has accumulated over a thousand years of such gestures, from Anglo-Saxon princes to Victorian scientists, each finding their place within walls that Viking raiders failed to destroy.