The fort was a product of Victorian Britain's most expensive case of nerves. Lord Palmerston had become convinced in the late 1850s that Napoleon III was about to invade, and the Royal Commission of 1859 recommended a ring of coastal fortifications around every important naval anchorage in the country. Portland Harbour - then becoming the principal Channel Fleet base - was top of the list. The first contractor went bust trying to level the Nothe Peninsula and build the sea wall, so the 26 Company Royal Engineers took over in 1862. The upper earthen parapet was finished by 1869. The whole D-shaped complex was commissioned in 1872 at a total cost of £117,049. By then Napoleon III had been dethroned and the threat that had inspired the fort no longer existed. The buildings of Britain's invasion panic became known to a later generation as Palmerston's Follies.

Inside the D, the engineers arranged three storeys for the business of artillery. The deep magazines held the powder. The ground-level casemates held heavy muzzle-loading cannons - two 64-pounders, four 9-inch rifled muzzle loaders, six 10-inch guns - and the soldiers who served them. The ramparts above could fire muskets and lighter guns during a close assault. In the 1890s, seven of the original guns were replaced with even heavier 12.5-inch RML pieces. Then breech-loading technology arrived, and at the start of the twentieth century three 6-inch Mark VII naval guns took their place on the ramparts. Hoists were installed to lift the 100-pound shells from the magazines below. The fort never fired at an enemy ship. Not once in 130 years of military service.

By 1938 the army had given up on coastal artillery and converted Nothe into a central anti-aircraft ammunition depot for the southwest of England, complete with electric hoists and a purpose-built loading platform. During the Second World War the fort got its only sustained military use: four Vickers QF 3.7-inch AA guns on a new platform in the northwest corner, later replaced by a Bofors 40mm. From this perch the gunners watched Luftwaffe raids on Portland Harbour and Weymouth. The fort itself was never seriously hit. After 1945 it lingered for another decade as a coastal-defence relic and was finally abandoned in 1956, briefly serving the Royal Navy as a store and a degaussing facility before being sold to the local council in 1961.

The borough council had no idea what to do with it. Plans were floated to convert it into a luxury hotel - work actually began in 1971 before being abandoned. The building slid into vandalism and rot. In 1980 the Weymouth Civic Society took over the lease and began the slow restoration that has continued ever since, helped along by Manpower Services Scheme labour and, in the late 1980s, an unexpected new tenant: civil defence planners installed blast doors in part of the magazine level and turned it into a nuclear shelter for local administration during the closing years of the Cold War. The fort that had been built against the French had ended up sheltering bureaucrats against the Soviets. The lottery later contributed £1.8 million, English Heritage added more, and a paid team of five with seventy volunteers keeps the place running today.

In 2007 a National Lottery survey voted Nothe one of the spookiest locations in the United Kingdom - a reputation that the bomb-proof casemates, the buried magazines, and the long quiet decades of neglect have all conspired to earn. Then in 2012, the Olympics came to Portland Harbour. The fort and the adjacent Nothe Gardens were closed to the public for a fortnight; instead, 150 marshals were recruited from hundreds of applicants, and 4,600 ticketed spectators a day climbed up to the fort each morning to watch the sailing events from the official viewpoint. Britain's Victorian paranoia had produced a building that ended up doing what nobody designed it for: serving as the world's most fortified grandstand for an Olympic regatta.