The Ker family, who became the Earls and later Dukes of Roxburghe, have held lands in Roxburghshire since the twelfth century. Their origins are not entirely certain, though they likely came over with the Normans. Since Sir James Innes became the 5th Duke in 1812, the family has used the hyphenated name Innes-Ker. The name Floors itself is uncertain. It may come from flowers, or from the French fleurs, or from the floors, meaning the terraces, on which the castle is built. The lands themselves belonged to the monks of Kelso Abbey across the river until the Reformation, when James VI handed them to Robert Ker of Cessford, who would become the first Earl of Roxburghe in 1616. Whether an earlier tower house stood on the site before the present building was raised is uncertain, but it is plausible. The Borders were tower country.

John Ker, 1st Duke of Roxburghe, had played a quiet but important role in securing the Union of England and Scotland in 1707. His reward was a dukedom in 1707, and his ambition was a country house worthy of it. He commissioned William Adam, the Scottish architect whose son Robert would later become the most celebrated designer in the English-speaking world, to design a new mansion. Adam worked between 1721 and 1726, producing a plain block with towers at each corner and flanking pavilions for stables and kitchens. By the early nineteenth century, that classical restraint had fallen out of fashion. The 6th Duke commissioned William Playfair, then at the height of his powers in Edinburgh, to give the house the look of a romantic castle. Playfair added the turrets, water spouts, and battlements that now make Floors look, from a distance, like a fortress on the Tweed. From a distance only. Up close, the lines stay too symmetrical, too regular, too obviously built for living rather than for fighting.

Floors stands almost directly opposite the site of Roxburgh Castle, today little more than a green mound. Roxburgh was, in the twelfth century, one of the most important strongholds in Scotland and a royal residence. In 1460, James II of Scotland besieged it during a campaign to recapture the castle from the English. The campaign succeeded, but James himself did not survive it. While he was inspecting his artillery, one of his own bombards, the great iron guns of the age, exploded and killed him. His widow Mary of Guelders ordered the castle demolished so that no English garrison could ever hold it again. The royal burgh of Roxburgh, once one of the four great burghs of medieval Scotland, dwindled to nothing. Floors Castle was built three centuries later in plain sight of those ruins, on the lands the dukes had received from a king who had no further use for them.

Floors Castle is now a Category A listed building, the highest tier of protection in Scottish heritage law. The Innes-Ker family still occupy the house, with the 10th Duke of Roxburghe, Charles Innes-Ker, succeeding his father Guy in 2019. The castle is open to the public seasonally and includes a walled garden, parkland along the Tweed, and views across the river to the ruins of Roxburgh Castle and the spires of Kelso Abbey beyond. The contrast is striking. On one bank, a working ducal seat in nineteenth-century pomp. On the other, a low green mound where a king died and a vanished royal burgh once bustled. The Tweed runs between them as calmly as if none of it had happened.