The name Cong comes from the Irish Cunga Fheichin, the narrows of Saint Feichin, a sixth-century Irish saint whose monastic foundations dotted the west of Ireland. The narrows in question are the isthmus between Lough Corrib and Lough Mask, the two largest lakes in this part of Connacht. The water between them flows almost entirely underground, through the porous limestone, leaving the surface peculiarly dry for land between two great lakes. The 1111 Synod of Rath Breasail included Cong among the five dioceses approved for Connacht, recognising its religious importance. In 1152, the Synod of Kells reorganised the church and merged Cong's territory into the archdiocese of Tuam, ending its formal diocesan status. The Catholic Church still lists Cunga Feichin as a titular see, a phantom diocese carrying the name forward.

Rory O'Connor, in Irish Ruaidri Ua Conchobair, was the last man to hold the title of High King of Ireland in any effective sense. His reign coincided with the Norman invasion of the late twelfth century, and he spent decades trying to manage a political order that was collapsing around him. In 1186, he retired from Tuam to Cong Abbey, where the Augustinian canons received him. He returned briefly to politics twice afterwards, but the High Kingship was effectively over. He died at Cong in 1198 and was buried there. The abbey he retired to is now a roofless ruin, but it remains one of the most atmospheric monastic sites in the west of Ireland, with surviving cloister arcades and the elaborate carved doorways for which the Augustinians were known. The Cross of Cong, a twelfth-century processional cross of extraordinary Celtic metalwork, is now in the National Museum of Ireland in Dublin.

Just outside Cong sits Moytura House, the lakeside summer residence built by Sir William Wilde, the prominent Anglo-Irish surgeon and antiquarian. Wilde was a serious historian of the west of Ireland, writing the comprehensive Lough Corrib, its Shores and Islands in 1867, and he brought his family to Moytura for long summers. His son Oscar Wilde spent boyhood and young adult summers in this house. The future playwright and aesthete walked the same lake shores that the Augustinians had walked, fished the same waters, listened to the same Connacht Irish spoken in the village. Moytura is now a private home, invisible from the lane, but the connection persists. The local poet Micheal Mac Suibhne, called the Bard of the West, was born at Cong around 1760 and composed his work in Connacht Irish before dying in poverty in Connemara around 1820. The literary lineage runs deeper than the film tourism would suggest.

On the Cong side of Lough Corrib stands Ashford Castle, originally a thirteenth-century fortification of the de Burgo family, later transformed by the Guinness brewing dynasty into a Victorian baronial fantasy. Sir Benjamin Guinness acquired the estate in 1852 and rebuilt it on an extravagant scale. His son Arthur Guinness, 1st Baron Ardilaun, extended the grounds further. After the Guinness era, the castle passed through various owners before becoming the luxury hotel it is today. Much of The Quiet Man was filmed on its grounds. Ashford Castle is now one of Ireland's most expensive hotels, but it is also a continuation of the building's long history as a country house for those who could afford to live by a lake.

The Anglo-Norman conquest stripped Cong of its centrality, and for almost eight centuries afterwards, very little happened. The village remained small. The abbey decayed. The traditional eighteenth-century cottages were not replaced. The streets stayed where they had been. When John Ford arrived in 1952 looking for a location for The Quiet Man, the village was preserved by neglect into exactly the look his romantic story required: lush, idyllic, rural Ireland of an imagined past. The film's success made Cong a tourist destination, and the village has lived off it for more than seventy years. The Quiet Man Fan Club still meets. Pat Cohan's Bar, named for the fictional pub in the film, operates as a real pub today. The bridge from the film, on the road between Oughterard and Maam Cross, is a regular pilgrimage stop. Cong learned to perform itself, and it has been doing so ever since.