Robert the Bruce rewarded Sir James's loyalty with vast holdings, and the Douglases became one of the great noble houses of Scotland. In 1357 Sir James's nephew was created the first Earl of Douglas. In 1384 his illegitimate son Sir Archibald Douglas became the third Earl, and the castle at Douglas was rebuilt as a primary stronghold of the family. Through the next century, the 'Black' Douglases grew so powerful they came to threaten the Stewart monarchy itself. In 1455 King James II marched against the rebellious ninth Earl and crushed his forces at the Battle of Arkinholm. The castle was sacked. The lands and titles were forfeited. They passed to the 'Red' Douglases, the Earls of Angus, who had sided with the king — and who, by some accounts, may have been just as glad to see their senior cousins humbled. The castle was rebuilt, of course. They always were.

In 1703 Archibald Douglas became the first Duke of Douglas, with his principal seat at the castle. A new tower house with an enclosed courtyard went up, and a corner tower at its edge — the same tower that still stands today. In 1755 fire swept through and destroyed everything except that one tower. From 1757 the Duke commissioned the Adam brothers — James, John, and Robert, the most famous architectural family in eighteenth-century Britain — to design something extraordinary. The plan was a five-storey castellated mansion with round towers on the front, square towers on the rear, and an enormous park spanning the valley of the Douglas Water. Had it been completed it would have been the largest castle in Scotland. But the Duke died in 1761, and only about half the design was ever built. A famous legal dispute called the Douglas Cause then erupted between his nephew and the Duke of Hamilton over who would inherit. The nephew won, was ennobled as Baron Douglas in 1790, and the estate passed through his daughter and granddaughter to the Earls of Home.

By the 1930s Lanarkshire was a place of catastrophic unemployment. Coal pits were closing, men with families had no work, and Charles Douglas-Home, the thirteenth Earl of Home, faced a choice that would have been agonising for any landowner with a sense of duty to his tenants. He allowed coal mining in the park immediately adjacent to the castle. It was a way to provide jobs — desperate, necessary jobs — but the mining caused subsidence. The ground beneath the great Adam-designed mansion began to shift. By 1938 the structure was dangerous, and it had to be demolished. The decision is hard to second-guess. The castle was a single building; the unemployment was a whole community.

Today only the seventeenth-century corner tower remains, three storeys and nine metres tall, standing on a rise to the south of the river. It was retained as a garden folly when the great mansion was built around it. Below, a small cellar block survives with glazed tiles still on the interior walls. Nothing visible remains of the Adam mansion itself. Sir Walter Scott used the castle and its early history as the inspiration for his novel Castle Dangerous, and locals still sometimes use that name. A former lodge from the estate has been preserved at the Cairn Lodge service area on the M74 — for many drivers heading south, an unremarkable petrol stop that is in fact the last fragment of a great family seat. The remains are protected as a Category C listed building. The Earls of Home, who became Douglas-Home, still hold the family name; one of them, Sir Alec Douglas-Home, briefly served as British Prime Minister from 1963 to 1964.