Tradition tells that Ninian dedicated his church to his teacher, Martin of Tours, the great 4th-century Gallic bishop whose example shaped early Western monasticism. When Ninian died, probably in 432, he was buried in the building he had founded, and that grave became, over the next thousand years, one of the most important pilgrim destinations in Britain. Around it grew a medieval cathedral, the priory whose roofless nave still stands on Bruce Street. James IV and James V of Scotland came on repeated pilgrimage; so did Margaret Tudor and Mary, Queen of Scots. The 1st-century settlement at Rispain Camp, about a mile west of the town, suggests that people were drawn to live in this corner of the Machars long before any saint arrived.

The town has more than one church to tell its story. St Martin and St Ninian Catholic Church, consecrated in 1960, stands on George Street. After the reforms of the Second Vatican Council its interior was reordered: the altar was brought forward from the east wall, the baldacchino was demolished, the ironwork at the baptistry and communion rail was scrapped, and the original Creetown granite altar was placed outside against the east elevation. A Hew Lorimer crucifix carved into the east wall has not fared well; a poorly handled cleaning in 1997 caused significant loss of detail. The church, including its quadrant walls, is listed Category C(S). Inspiration for the design may have come from San Julian de los Prados in Asturias, Spain. Just up the road, the 1822 St Ninian's Priory Church of Scotland was built with stones taken from the medieval priory itself.

Whithorn's link to the sea was the Isle of Whithorn, three miles to the south-east. Though called an island, it is now a peninsula, and was the harbour through which medieval pilgrims arrived. The 13th-century St Ninian's Chapel on Isle Head marked the spot where pilgrims first stepped onto Galloway soil. From there, a 25-mile signposted Pilgrim's Way, kept up today by the Whithorn Pilgrimage Trust, ran inland through the Machars from Glenluce Abbey. Walking it now you can still feel the rhythm the pilgrims walked: from sea to shrine, with stops at the holy sites along the way. The medieval church survived as cathedral of Galloway until the Reformation.

Whithorn's residents past have a remarkable reach. Charles Lockhart, born 1818 at Cairnhead in the Isle of Whithorn, went to America and in 1874 merged his Pittsburgh refineries into the Standard Oil Company, becoming its president and one of its four key founders alongside John D. Rockefeller, Henry M. Flagler, and William G. Warden. Alastair Reid, the poet and translator born in Whithorn on 22 March 1926, became the English voice of Jorge Luis Borges and Pablo Neruda; in his later years he and his wife Leslie kept a cottage on the Galloway House estate. George Dickie, born and raised here in 1912, fought in the Spanish Civil War from 1936 to 1938 under the nom de guerre Jack Brent, against Franco's forces; a memorial plaque to him was unveiled in 2006 on the former butcher's shop next to The Pend where he once worked. And Jeannie Donnan, 'The Galloway Poetess' who lived from 1864 to 1942, wrote local poetry from her George Street home, also marked now by a plaque.