The name Hugh comes from the Old English word hoh, meaning a promontory or piece of elevated ground. The same word survives in Plymouth Hoe across the water in Devon. The peninsula now called the Garrison was, in earlier centuries, simply The Hugh, and the town that grew at its eastern foot eventually borrowed the name. During medieval times the population centre of St Mary's lay at Old Town, on the south coast. Hugh Town only began to grow in the 17th century, after Star Castle was built on the Garrison in 1593 and the harbour quay was completed in 1601. The military fortifications kept expanding through the 18th century; The Hugh became The Garrison, and the village beneath it became the islands' principal port.

Almost everything else in the Isles of Scilly belongs to the Duchy of Cornwall. Hugh Town does not. In 1949 the Duchy sold the freehold of the town to its inhabitants, an event still recounted as the moment the islanders became, on paper, masters of their own front doors. The harbour itself remained Duchy property, and is run today by the St Mary's Harbour and Pilotage Authority. The quay was extended by another 23 metres in 2015 to accept deeper-draught vessels, and a boardwalk was added in 2021 to improve pedestrian access. The Scillonian III ties up here through the season, joined by the cargo vessel Mali Rose. Both are registered at St Mary's.

On the first weekend of May, Hugh Town fills up. Crews arrive from across Britain to row in the World Pilot Gig Championships, an annual race between fast six-oared boats that descend from the working gigs once used to ferry pilots out to incoming ships. The competing teams base themselves in the bays around St Mary's Pool, and the finishing line for the major races sits at the end of the harbour quay. Spectators line the walls. The harbour fills with gigs of every paint scheme imaginable. Crews from Penzance, Plymouth and as far afield as New York and Australia have come for it. For one weekend a year, a town of 1,097 souls hosts thousands.

When new housing went up on the Porthcressa side of Hugh Town in 1949 and again in 1960, the builders dug up cemeteries containing the remains of native Britons dating from the 1st to the 4th century AD. A Roman altar was discovered on the Garrison in the 19th century and now sits in the Tresco Abbey Gardens, hinting that the headland once carried a temple or shrine that has never been excavated. The isthmus people walk across between two beaches has been a settled place for at least two thousand years. The current town hall, built in 1889 of squared granite, replaced the older parade ground for the Sea Fencibles, the Napoleonic-era coastal volunteers who drilled here against a feared French invasion that never came.

Hugh Town has one of the mildest climates in the United Kingdom. Temperatures rarely dip below freezing; the most recent serious snowfall was during the 2018 cold wave, and even that was modest. The Azores High nudges summer weather towards a dry trend, and the surrounding Atlantic keeps winters wet, windy, and tepid rather than cold. Carn Thomas, the rocky outcrop at the east end of Town Beach, has been the home of the St Mary's Lifeboat Station since 1837. The lifeboat crews still launch from there into the same seas that wrecked Cloudesley Shovell's fleet in 1707 and the MV Cita in 1997, both within a few miles of the harbour wall.

Coordinates: 49.9146°N, 6.3160°W. St Mary's Airport (EGHE) sits one mile east of the town centre, on the high ground above Old Town Bay. Approach from the west at 2,000 feet and Hugh Town reveals itself as a strip of slate roofs on a pale sandbar between two arcs of beach. The Garrison peninsula on the western side, with its star-shaped Elizabethan castle, is the strongest visual anchor; the quay extends north into St Mary's Pool. Visibility shifts quickly here; check actuals at EGHE before commitment.