Cromwell's 1539 proposal went nowhere. The Tudor state had other priorities. In 1748 the Welsh polymath Lewis Morris surveyed Milford Haven for shipwrecks and navigation hazards, and recommended a small fort on Stack Rock. Nothing happened. A further survey in 1817 made the same recommendation. Nothing happened. The pattern held for over two hundred years - everyone agreed the island was the perfect spot to defend the haven, and nobody quite got around to doing it. What finally changed the calculus was Napoleon III, who took power in France in 1851 and spent the 1850s rattling sabres at Britain in a way that worried Parliament enough to fund what came to be called the Palmerston Forts - a vast chain of coastal defenses built under Prime Minister Lord Palmerston in the 1860s. Stack Rock became part of that chain. In 1850, even before the panic peaked, work began on a three-gun tower on the island.

The first structure was an elliptical stone tower, 55 feet across at its widest and 30 feet tall, mounting three 32-pounder guns and a 12-pounder for close defence. Construction took two years, finishing in 1852. The tower was barely complete before it became obsolete. In 1858 the French launched the ironclad warship La Gloire, with armour plating that could shrug off the 32-pounders the new tower carried. Britain's coastal defenses had been instantly outdated by a single ship. The Royal Engineers' response was to build a much larger fort - and to wrap it completely around the original tower, which still survives buried inside the later structure like a smaller Russian doll. The expansion was redesigned three times during construction. Initial plans called for fifty-four guns on two decks. Then granite piers replaced limestone. Then the design shrank to sixteen large guns in a single ring of casemates facing the seaward approaches, with seven smaller guns covering the landward gorge. The top floor became accommodation for four officers and 152 men. Three roof turrets, each meant to hold two huge 25-ton rifled muzzle-loaders, were planned but never installed. The fort was completed in 1871 - twenty years after work began.

The French invasion that justified all this expense never came. Napoleon III was defeated by the Prussians in 1870, and his successors had other things to think about. The fort's big rifled muzzle-loaders were replaced in 1902 with four 12-pounder quick-firing guns. During the First World War a small garrison kept watch with just two of the QFs remaining, paired with a system of remotely-controlled anti-ship mines in the haven and searchlights to sweep the channel against night raids. No raid ever came. In 1929 the War Office decommissioned the fort entirely. Three years later, in 1932, it was put on the market for the first time. It sold for 160 pounds - a sum that would have barely covered the cost of replacing a single gun carriage.

For most of the twentieth century Stack Rock Fort sat empty in the middle of the haven. Birds nested in the casemates. Vegetation grew between the granite. Concerns surfaced periodically about trespassers - in 2013 the proximity to a nearby LNG jetty made unauthorised access a real security worry. The fort changed hands again in 2020, sold to a private owner for an undisclosed price. In January 2021 the Land Registry recorded a further sale, to a community interest company called Anoniiem, for 191,000 pounds. The trust's plan is gradual restoration without erasing the patina of a century's abandonment - to make the fort visitable again without pretending the long quiet years never happened. From boats on the haven, the fort still looks formidable: a low grey crown of stone rising from the water, slit-windows facing the channel as they were designed to. Up close it is weathered, ivy-clad, full of seabirds. A monument to a panic that proved unfounded, built so well that it has outlasted both its purpose and its threat.

Stack Rock makes more sense from the water than from anywhere else. The fort sits roughly in the middle of the haven's lower reach, where the channel narrows between Angle Point to the south and the headlands above Sandy Haven to the north. Any ship entering or leaving the haven has to pass it. That is the whole logic of the place: a single fort, on a single rock, controlling a single line. The same logic applied to the Martello towers built at the same time on the dockyard's corners across the haven at Pembroke Dock - and to the two-thousand-year-older fortifications at Pembroke Castle further up the estuary. People have been trying to control this channel since the Iron Age. They are still trying. The LNG tankers from Qatar that now glide past Stack Rock pay the same attention to the same rocks the Victorian gunners did, for the same reason.