The original site is older than anyone can quite pin down. The Irish name was Rath Fhionnáin — Finan's ringfort — and the earthwork dates to the 7th or 8th century AD, though similar structures across Ireland go back further still. A souterrain (an underground stone passage, used for storage or refuge) opens into the ground on the southeast side. The entrance to the enclosure faces southwest. Local tradition once claimed this was the last piece of land in Ireland held by the Vikings, because the ditch made it so easy to defend; the historical record does not confirm this, but the legend tells you something about how impressive the earthwork looked even to people who knew nothing about who built it.

The FitzGeralds came to Ireland with the Normans and stayed. Three branches of the family held hereditary knighthoods: the White Knight, the Knight of Glin, and the Knight of Kerry. The Knights of Kerry built Rahinnane Castle as one of their tower houses — three storeys of rectangular stone, walls thick enough to absorb most of what 16th-century weaponry could throw at them. The blind arcades inside, the traces of vaulted ceilings, and the corner turrets all mark this as a working defensive home, not a romantic folly. Hundreds of similar towers were built across Ireland in this period by Anglo-Norman lords who needed to protect their families and their cattle from raids, feuds, and the unpredictable politics of the Tudor era.

The Nine Years' War (1593–1603) was the largest Irish rebellion against Elizabethan rule, led by Hugh O'Neill and Hugh Roe O'Donnell. The fighting reached Munster late in the war. In 1602, after the Battle of Kinsale had effectively broken the rebellion, English commander Sir Charles Wilmot moved through Kerry to mop up resistance. Rahinnane was one of the strongholds he took. The records do not say what happened to whoever was inside — Tudor accounts of these reductions are often brisk and brutal. The castle came under English control, and the Knight of Kerry's family saw their hereditary seat slip from their grip.

Fifty years later the castle was finished off. Oliver Cromwell's conquest of Ireland between 1649 and 1653 was an attempt to reduce the country militarily and demographically — a campaign that historians estimate killed somewhere between fifteen and twenty percent of the Irish population through war, famine, and disease. Tower houses across the country were systematically slighted: roofs pulled down, doors burned, walls breached so they could not be used again. Rahinnane was one of them. The Cromwellians left more than half the outer walls standing, though, and that is what remains today — three storeys of weathered stone rising from a 9-metre ditch that predates them by a thousand years. The two stories sit on top of one another like geological strata: ringfort and tower house, both ruined, both still recognizable.