The courthouse and jail were authorised by an Act of Parliament passed in the 54th year of George III's reign, to replace an earlier town house on Front Street that dated from 1755 and had become too small. James Gillespie Graham produced the 1813 design that was eventually built, after Robert Reid's original plans had been abandoned for lack of funds. The courtroom on the first floor is shaped as a hemicycle with large windows overlooking the prison yard and the waters of Loch Fyne beyond. It is a striking room, deliberately impressive, the kind of space designed to make the workings of the law feel ordered and serious. The whole building is now a Category A listed structure, the highest tier of architectural protection in Scotland.

The Prisons (Scotland) Act 1839 changed Inveraray. The Act required the separation of prisoners by category, and to make this possible a new three-storey prison block was added to the southwest of the courthouse, designed by the prison architect Thomas Brown in coursed rubble and completed in 1845. The new block had individual cells, gas lighting, and a system of separation that aimed, in the language of the period, to give prisoners time alone to reflect on their offences. The reformers of the day believed solitude promoted moral change. We now understand it differently: long periods alone in small cells were a form of severe psychological pressure, applied to people including children and the mentally ill. The new block is preserved today as part of the museum, and visitors can step into the cells.

The Prisons (Scotland) Act 1877 transferred prison management from local authorities to central government and pushed for the closure of smaller prisons in favour of larger institutions in the cities. The opening of HM Prison Barlinnie in Glasgow in 1882 made Inveraray's role redundant. The remaining prisoners were transferred to Barlinnie, and Inveraray Prison closed in 1889. The courthouse, however, kept working. It hosted meetings of the Argyll Commissioners of Supply, the main county administrative body before elected county councils were established under the Local Government (Scotland) Act 1889. The new Argyll County Council held its first meeting at Inveraray on 22 May 1890, and after more than three hours of debate, decided to meet in Dunoon for the summer and Oban for the winter, with main offices at Lochgilphead.

By the mid-twentieth century the courthouse had become dilapidated. In 1954 an order ended sheriff court hearings at Inveraray; Justice of the Peace hearings continued for a while longer. Argyll County Council sold the building in 1962. A refurbishment by Ian Gordon Lindsay and Partners in 1965 prepared it for a new life, and in May 1989 it opened as a visitor attraction. The museum now uses the courtroom and cells to tell stories of specific prisoners: their names where the records survive, the small offences that brought many of them inside, the sentences they served, and the deaths some of them met in custody. The cell doors swing open today for visitors. They remain a reminder that for the people who lived behind them, in cold and damp and crowded conditions, the doors did not swing open at all.