King Stephen's charter gave his lands in the fields of Norwich to the Church of St. Mary and St. John, to be held as freely as the king himself held them. Upon this grant, Seyna and Lescelina — two nuns who were sisters — began construction of the priory, dedicating it to St. Mary of Carhowe. The Benedictine house was founded for a prioress and nine 'black nuns', though the community later grew to twelve.

Among the notable figures associated with the priory is the anchoress Julian of Norwich, said to have received her religious training here in the 1350s or 1360s. Her writings show distinctly Benedictine characteristics, and her later mystical work — including the Revelations of Divine Love — has its roots in this tradition.

The priory had its conflicts. In 1414, a dispute broke out between the Prioress of Carrow and the Prior of Holy Trinity over ownership rights. Prioress Edith Wilton found herself accused of harbouring a murderer; she counter-accused the monks of driving away her cattle. She won the litigation and enjoyed the backing of prominent Norwich citizens who helped secure her release on bail during the dispute. By October 1419, the two houses had made peace and agreed the limits of their respective jurisdictions. In 1538, the priory was dissolved. The last prioress, Cecily Stafford, received a pension of five pounds a year.

After dissolution, the site passed through several owners before reaching the Martineau family, who could trace their descent from a Huguenot refugee — one of the French Protestants who fled religious persecution on the Continent and found a haven in the tolerant religious climate of Norwich. This distinguished medical family purchased the estate and in 1847 their property was described as 'a handsome mansion with pleasure grounds delightfully laid out.'

From the ruins of the old priory buildings, the Martineaus also constructed on their estate a 'small gothic priory with windows of ancient stained glass' — a fashionable folly that was itself built from the fragments of the medieval original. The estate changed hands again in 1879, passing to J & J Colman, the mustard and condiment firm whose name still appears on millions of kitchen tables.

Jeremiah James Colman and his wife Caroline raised their six children at Carrow Abbey. Their daughter Ethel Colman later became the first woman Lord Mayor of Norwich. The family hosted the Prince of Wales for a lavish luncheon in July 1900 and undertook considerable renovation work between 1899 and 1909, adding a new wing and the hooded stone fireplace that bears the date 1900 in the entrance hall.

The building that survives today is a layered structure: foundations from 1146 beneath brickwork mostly from the 16th century beneath renovations from the late 19th century. The entrance hall, the moulded plaster ceilings in the right wing, the 16th-century panelling in the parlour, a Gothic-style staircase with crockets and lion finials — these details accumulate into something that feels genuinely old, genuinely inhabited. Carrow Abbey was listed as a Grade I listed building in 1954. It is not open to the public as a museum, but it stands, still legible as the place where two sisters began building something in 1146.