Elgin first appears in the Cartulary of Moray in 1190, but the place was older than the documents. The Elgin Pillar, a 9th-century class II Pictish stone, was dug up under the High Street in 1823, suggesting Christian presence in the area long before the burgh existed. King David I made Elgin a royal burgh around 1130 after defeating Oengus of Moray, and a castle was raised on Lady Hill at the west end of what is now the High Street. In August 1040, somewhere near here at Bothganowan, MacBeth's army defeated and killed Duncan I. The chroniclers placed the killing at Pitgaveny, just outside town. Shakespeare's version with the moving forest of Birnam was 600 years and one country away. The real killing happened in Moray.

On 19 July 1224, the foundation stone of Elgin Cathedral was laid under royal charter from Alexander II. The building was completed sometime after 1242, burned in 1270, and rebuilt — what stands today dates mainly from that reconstruction. By 1390 it was magnificent enough that the Chartulary of Moray described it as the mirror of the country and the glory of the kingdom. Then Alexander Stewart, the Wolf of Badenoch, descended from his island stronghold at Lochindorb. He had been pulled from his role as the bishop's protector and was angry. In May he burned Forres. In June he burned much of Elgin, including two monasteries, St Giles Church, the Hospital of Maison Dieu, and the cathedral. Andrew of Wyntoun described the destruction as the work of wyld, wykked Heland-men. Rebuilding took decades. Much of the later work has crumbled because of inferior stone; the 13th-century construction still stands.

After the Reformation, the cathedral had no defenders. In 1568, the Privy Council ordered the lead stripped from the roof to fund Regent Moray's soldiers. The ship carrying the lead to Holland sank almost immediately on leaving Aberdeen harbour, which the people of Elgin took as a sign. Without the roof, the building began to die in earnest. In 1637 the rafters over the choir blew down. In 1640 Covenanters from the local Kirk Session destroyed the ornate carved screen and woodwork. Cromwell's soldiers smashed the tracery of the west window between 1650 and 1660. On Easter Sunday 1711, the central tower collapsed for the second time, this time catastrophically. The rubble was quarried for local construction until 1807, when a wall was finally built around the site and a keeper hired. John Shanks cleared the debris by hand, revealing the stonework, and the authorities gave him an ornate snuffbox now kept in Elgin Museum.

Elgin sits on rare Permo-Triassic sandstone, formed when this region was desert. A quarry on the town edge called Cuttie's Hillock produced fossils internationally known as the Elgin Reptiles — including Ornithosuchus, a 2-metre crocodile that walked on its hind legs across the Triassic sands. The Elgin Museum preserves these creatures, which is one of those small surprises that small Scottish towns specialise in. The other famous Elgin name has nothing to do with the town. The Elgin Marbles, removed from the Acropolis by the 7th Earl of Elgin in the early 19th century, were named for his title, not his residence. The Earls of Elgin lived in Fife. None of them lived here.

Elgin became unexpectedly central to the 1960s British music circuit. The Two Red Shoes dancehall hosted the Beatles, the Who, Pink Floyd, Cream, and Dusty Springfield. The Kinks played Elgin Town Hall. Today the town's economy depends heavily on whisky — Glen Moray, Miltonduff, and BenRiach distilleries all sit within six miles — and on the RAF. RAF Lossiemouth is the largest fast-jet base in the UK. In 2005 the local RAF stations contributed £156.5 million to the Moray economy. Prince Charles Edward Stuart slept in Mrs Anderson's house at Thunderton for 11 days in March 1746, before riding south to Culloden. She kept the sheets he slept on and was buried in them. Elgin remembers the people who pass through it.