Saint Cadfan stepped ashore here sometime in the early sixth century, a Breton missionary who saw in Bardsey what later pilgrims would also see: a place sufficiently far from the world. He founded a community on the high ground at the island's northern end, where freshwater springs surface and the cliffs of Mynydd Enlli shoulder off the worst of the Atlantic gales. Around him gathered other monks, and around them gathered legends. Dubricius, the elderly Archbishop of Caerleon who had retired in favour of the younger Saint David, is said to have ended his days here too. Within a century or so, Bardsey had become Ynys Enlli, the Island in the Currents, a name that warned sailors and beckoned the devout in roughly equal measure.

By the High Middle Ages, Bardsey had become one of the three great Welsh pilgrimage destinations, alongside St Davids in Pembrokeshire and Holywell in Flintshire. The phrase was widespread enough that medieval Welsh travellers held three journeys to Bardsey to be the spiritual equivalent of a single pilgrimage to Rome. Gerald of Wales, riding through in 1188 with Archbishop Baldwin to drum up support for the Third Crusade, paused long enough to note in his Itinerarium Cambriae that the monks here lived as a community of "very religious men." Almost a century later, in 1284, Edward I made the crossing as he toured his newly conquered Welsh territories. By then the Augustinian Abbey of St Mary, successor to the original Celtic foundation, had risen on the same hallowed ground.

Henry VIII's Dissolution of the monasteries reached even here in the 1530s, scattering the abbey's community and beginning the long slow process of collapse. In 1821, the topographer Peter Bayley Williams could still describe substantial walls and recognisable vaults in his Tourist's Guide through the Country of Caernarvon. By the time photographers from the Royal Commission on the Ancient and Historical Monuments of Wales began documenting the site in the early twentieth century, only the tower stub remained, a roofless thumb of masonry pointing nowhere in particular. Local farmers had spent four centuries hauling abbey stone for cottages, barns, and field walls. The Grade II*-listed remnant is now scheduled as an ancient monument, alongside three Celtic crosses set among the grass and the buried foundations.

The claim of twenty thousand burials predates archaeology and resists it. Yet excavations and ground surveys have repeatedly turned up human remains across the island's slopes, and the Bardsey Island Trust still occasionally encounters bone when digging fence posts or repairing paths. Whether the saints arrived alive seeking sanctuary, or dead seeking holy ground, or whether the figure simply means "a great many," matters less than the fact that for a thousand years Welsh families thought it worth the crossing. The tide here runs fierce through Bardsey Sound, sometimes too fierce for the modern ferry. The Welsh called the strait Ffrydiau Caswennan, a hint that the journey itself was always part of the pilgrimage.