The first building on the site went up between 1283 and 1291, when Thomas Bek was Bishop of St Davids and a college of priests was established here. In 1542, Bishop William Barlow decided the diocese needed a more comfortable seat than the windswept clifftop at St Davids itself. He shifted the bishop's palace inland to Abergwili, reusing the old college buildings. Bishops of St Davids would live here, in some form, for the next 432 years. The palace burned down almost entirely in 1903 and was rebuilt the same year. It still contains the chapel that Archbishop William Laud, then Bishop of St Davids, added in 1625. In 1974 Carmarthenshire County Council bought the old palace from the church and turned it into the county museum. A new, more modest bishop's residence, Llys Esgob, was built in the grounds. The connection has now lasted well over four centuries, and the diocese has not yet broken it.

In 1563 Parliament ordered the Welsh and Hereford bishops to produce a translation of the Bible, the Book of Common Prayer, and the sacraments in Welsh by 1 March 1567. The work fell to William Salesbury, a Renaissance scholar with a gift for languages, and to Bishop Richard Davies, who installed Salesbury and a small team at his Abergwili palace. Davies translated several of the shorter epistles himself. The precentor Thomas Huet took on Revelation. Salesbury did the rest, working from the Greek. The first Welsh New Testament was published on 7 October 1567. The Welsh Bible that would follow in 1588, William Morgan's full translation built on this foundation, gave the Welsh language a literary spine at exactly the moment it might have splintered under English pressure. Tudor and Stuart Wales did not have great cities or printing houses of its own. It had a few country palaces with scholars in them, and one of those palaces was here.

Just outside the village rises a long whaleback ridge crowned with the remains of an Iron Age hillfort. Locals know it as Merlin's Hill, Bryn Myrddin in Welsh. The legend, recorded in medieval Welsh tradition and embroidered ever since, holds that this is where Merlin is buried, or imprisoned, or sleeping, depending on which version you choose. A path climbs from the small visitor centre at the foot to the summit, where the ramparts of the old fort are still legible in the turf. The neighbouring town of Carmarthen takes its Welsh name, Caerfyrddin, from the same association: Merlin's fort. Abergwili sits in the cradle of that old story, and on a misty morning, with the wooded ridge half-visible above the river valley, it is not difficult to see why the storytellers chose this hill rather than some other.

Until the great railway closures of the 1960s, Abergwili sat on the line running north from Carmarthen up the valley toward Aberystwyth. The station closed and most of the track came up, but a short stretch survived. Today the Gwili Railway, a volunteer-run heritage line, brings steam trains back along the old route through some of the gentlest scenery in west Wales. The hospital that serves the wider area, Glangwili General, sits at the village's western edge. The Wheat Sheaf pub, said to be one of the oldest in Carmarthenshire, still serves at the centre. Beyond it, the lily pond that used to be the River Towy itself, until the river jumped its banks in 1802 and cut a new channel half a mile south, hosts herons and dragonflies.

Notable people who died here include Alice Abadam, the Welsh suffragette and orator who spent her later years in the village before her death in 1940, and two consecutive bishops of St Davids, Basil Jones and John Owen. The Celtic cross in the churchyard remembers the village's dead from two world wars. The bilingual primary school in the centre teaches in Welsh and English, as the bishops' translators would have wished. Abergwili has grown a little in recent decades, but it has kept the shape of a riverside village built up around a single old institution, and that institution, in a quiet way, helped change the shape of a country.