Lady Richeldis de Faverches was an Anglo-Saxon noblewoman, a widow, who held her manor in Walsingham at the time of Edward the Confessor. According to tradition, she experienced a series of visions in which the Virgin Mary showed her the house at Nazareth where the Annunciation had taken place, and commissioned her to build a replica. Lady Richeldis' visions are among the earliest recorded Marian visions in England. The Holy House she built became a centre for prayer and pilgrimage.

In the twelfth century, the site was granted to Augustinian canons, who incorporated it into a newly established priory — the Augustinian Priory of The Annunciation of the Blessed Virgin Mary. This institutional foundation gave the shrine lasting stability. Walsingham received gifts of land, rents, and churches. Its canons recorded miracles. Pilgrims brought offerings. By the medieval period, the shrine's reputation extended across all of Christendom, drawing visitors that ranged from the destitute to the crowned.

The list of royal pilgrims to Walsingham is striking in its length. Henry III came in 1241. Edward I made the journey in 1280 and again in 1296. Edward II visited in 1315. Henry VI in 1455. Henry VII in 1487. And Henry VIII himself, who would later destroy the shrine, made the pilgrimage in 1513 — still a Catholic king visiting England's most sacred Catholic site.

In 1511, Erasmus made a pilgrimage from Cambridge in fulfilment of a vow. He left as his offering a set of Greek verses expressing his piety. Thirteen years later, in his Colloquies, he described the wealth and magnificence of Walsingham while rationalising some of its reputed miracles — a characteristic combination of admiration and scepticism from the great humanist scholar. The shrine's canons had received so many gifts that the community had grown wealthy beyond what a village priory might otherwise have expected.

In 1537, while the last prior, Richard Vowell, was making his accommodations with Thomas Cromwell and the approaching Dissolution, the sub-prior Nicholas Milcham was charged with conspiring against the suppression of the lesser monasteries. On flimsy evidence, he was convicted of high treason and hanged outside the priory walls.

In July 1538, Prior Vowell assented to the destruction of Walsingham Priory, assisting the king's commissioners in removing the figure of Mary and many gold and silver ornaments as part of the shrine's general spoliation. His reward for compliance was a pension of 100 pounds a year. The fifteen canons received smaller pensions of four to six pounds each. The site was sold to Thomas Sidney for 90 pounds. The figure of the Virgin was taken to London and burned.

The grief was immediate and widely expressed. The destruction of Walsingham was lamented in several Elizabethan ballads, including the widely-circulated Walsingham, also known as As I Went to Walsingham. Other works associated with the shrine's loss include A Lover's Complaint, attributed to Sir Walter Raleigh, and The Wreck of Walsingham.

The most prominent surviving architectural remnant of Walsingham Priory is the fourteenth-century East Window of the priory church, still standing in the abbey grounds. Over time, the former Prior's lodging was expanded and became a private mansion known simply as 'The Abbey.' The grounds of Walsingham Abbey and the adjacent Shirehall Museum remain open to the public.

The pilgrimage to Walsingham never fully ended. In the nineteenth century, pilgrimage to the site was revived, and today Walsingham remains an active Anglican and Roman Catholic pilgrimage destination, one of the most visited in England. The Palmers' Way still exists, still followed by those making the journey on foot. The Holy House is gone. The route to it is not.