Archibald Campbell, then Earl of Ilay and soon to become the 3rd Duke of Argyll, began work on the Inveraray estate in 1743. Three years later, in October 1746, the foundation stone of the new castle was laid. The political timing matters. The Jacobite rising of the previous year had just been crushed at Culloden, and Archibald Campbell was the dominant pro-Hanoverian power in the western Highlands. Building a fantasy castle here was a statement about the kind of Scotland the Campbells now intended to project: ordered, modern, allied with London, and confident enough to play with the visual language of an older, wilder Scotland it had helped to bring to an end. The new castle replaced an earlier fifteenth-century one on roughly the same site.

Step inside the castle today and the most striking room is the Armoury Hall, a soaring space whose walls are lined with more than 1,300 pikes, muskets, swords, and other weapons arranged in geometric patterns. The collection is part history, part decor: the polearms above the doorways form sunburst rosettes; the muskets are stacked in radial wheels around the ceiling. Below them hangs a portrait gallery of past Dukes and Duchesses. The castle is open to visitors during the season, and the present Duke and Duchess still live here. The current Duke and Clan Chief is Torquhil Campbell, the 13th Duke of Argyll, who succeeded his father in 2001.

The castle has had a second career as a film location. In 2012 it featured prominently in a Downton Abbey Christmas special, standing in for Duneagle Castle in the storyline that took the Crawley family north for a shooting party. It appeared again in the BBC mini-series A Very British Scandal, which dramatised the very public 1963 divorce case involving the 11th Duke of Argyll and his third wife Margaret. In 2024 it returned to screens in the second season of the Netflix series The Diplomat. The annual Best of the West festival, organised by the Duchess, was held in the grounds each September until 2018. The castle has also featured in Great Estates Scotland in 2014, Susan Calman's Secret Scotland in 2020, and An American Aristocrat's Guide to Great Estates.

Inveraray is reputedly haunted. The most often-told story involves a young Irish harper killed in 1644, when the army of the Marquess of Montrose sacked Inveraray and burned much of the surrounding territory. The musician is said still to play his harp in the MacArthur Room, audible particularly when a member of the Campbell family is dying. There is also a so-called Galley of Lorne ghost, supposedly seen sailing across Loch Fyne. Whether any of this is real, the loch view itself is the castle's quiet asset. Loch Fyne is Scotland's longest sea loch, narrowing to a head near Inveraray with the rounded green hills of Argyll Forest Park rising around it. The castle gardens sit close to the shoreline, and on a still evening the lights of the castle reflect across the water in a way that makes the Gothic Revival fantasy seem, for a moment, like the most sensible thing anyone ever built.