Mary Caroline was the second wife of the third Duke of Sutherland, whom she married in 1889. She was already known as Duchess Blair, after a previous marriage to a Highland Light Infantry captain who had been killed in a hunting accident near Pitlochry. The Sutherland family did not approve of her, and when the Duke died in 1892 his will - which left a great deal to her - was contested by his son and heir Cromartie. The court case did not go in the Duchess's favour. She was found guilty of destroying documents and spent six weeks in Holloway Prison. Eventually a settlement was reached. The family agreed to fund the construction of a castle for the dowager Duchess, on one strict condition: it had to be built outside the Sutherland lands.

She chose her ground carefully. A hill in Ross-shire, just over the boundary, looking directly across the Kyle of Sutherland at the family's traditional country. She hired Ayrshire builders, commissioned a Scottish baronial pile of three hundred and sixty-five windows and forty-one thousand square feet, and finished it between 1906 and 1917. The clock tower, with its three faces and one blank wall, became local legend almost immediately. Inside, the lower gallery filled with Italian marble statues and paintings dating to 1680. The dowager was not subtle. The castle was meant to be impossible to ignore from across the firth, a permanent visual reminder to the Sutherland family of the settlement they had paid. Today the building still has Category B listed status and remains one of the most theatrical houses in the Highlands.

During the Second World War, Carbisdale served a more dignified purpose. When the Nazi occupation of Norway forced King Haakon VII and Crown Prince Olav into exile, the castle became one of their refuges. On 22 June 1941 a meeting at Carbisdale established that if Soviet forces entered Norwegian territory in pursuit of the Germans, they would withdraw after the war. Three years later, in October 1944, the Red Army did indeed cross into northern Norway and capture about thirty towns. They later withdrew, exactly as the Carbisdale Conference had agreed. A castle built out of family resentment briefly became a quiet hinge of European diplomacy.

After the war, Captain Harold Salvesen inherited the castle from his father and made an extraordinary gift. He handed Carbisdale - the building, its contents, the estate - to the Scottish Youth Hostels Association. From 2 June 1945 until 2011, generations of walkers and cyclists slept in marble-floored halls under chandeliers and slept in rooms looking out at the Kyle. It was famously the grandest youth hostel in Britain. Decades of dormitory use eventually overwhelmed the building's fabric. The hostel closed for repair in 2011, and most of the art collection - 17 marble sculptures and 36 paintings - was auctioned in 2015, raising about a million pounds.

In 2016 the castle was bought by FCFM Group Ltd with plans for a 'world-class private residence'. By 2021 it was back on the market at £1.5 million. In October 2022 the barrister Samantha Kane bought it, took the title Lady Carbisdale, and began restoration with the intention of opening part of the house to the public. As of recent listings, it is once again for sale - the guide price now around £3.5 million. Locals still mention the ghost called Betty, who is said to haunt the corridors. Whether or not you believe in her, the castle has accumulated more than enough human drama in its short life to justify a few wandering spirits.