Shau Kei Wan grew up around the sea. Its earliest residents were fishermen who prayed to Tin Hau, goddess of the sea, and to Tam Kung — a deity particularly associated with Shau Kei Wan, believed to control weather and heal the sick. Temples to both still stand on Shau Kei Wan Main Street East, along with shrines to Shing Wong and Yuk Wong. The fishing industry has diminished, but the religious calendar hasn't changed.

Light industry arrived in the 1920s. After the Second World War, mainland Chinese refugees settled in large numbers, building around thirteen informal mountain villages on the hillsides above the bay. Most were slum housing with poor sanitation. In 1975, a large fire swept through slum structures in the Aldrich Bay typhoon shelter — dwellings built along and over the water. The government cleared the remaining informal settlements and replaced them with public housing estates. The coastline, already reshaped by landfill, was transformed again. It is said the shoreline today bears little resemblance to what existed in the late nineteenth century.

Shau Kei Wan Main Street East is the neighborhood's commercial spine and its oldest surviving street. In the nineteenth century it ran along the actual waterfront; when the government opened it up in the 1860s to displace the pirates who had been using the area as a refuge, shops and houses were built along both sides. Today, restaurants and food vendors occupy storefronts that have been handed down through families for generations, squeezed between modern high-rise residential towers.

When the street was widened, community pressure saved a hundred-year-old fig tree growing in the middle of the road. It still stands — a Ficus superba, designated a Champion Tree by the Hong Kong government. The other Champion Tree in Shau Kei Wan, a coconut palm, stands in front of the old Urban Council market. These two trees are as close to monuments as a working neighborhood like this gets.

At the eastern end of the neighborhood, near Lei Yue Mun, stands the Hong Kong Museum of Coastal Defence. The British built the original fort here in 1887, siting it to guard the eastern approach to Victoria Harbour. The fortifications were used for coastal defence through the Second World War, when the Japanese invasion reached Hong Kong. Today the museum covers the full sweep of Hong Kong's military history, with the old castle, defence basement, and military vehicles still accessible across 34,200 square meters of grounds.

The Tsung Tsin Church on Basel Road has a similarly layered history. Built in 1862 by local Hakka Christians and Swiss Basel missionaries, it was rebuilt in 1933, and in 1941 the Japanese military briefly used it as temporary headquarters for military police. It was rebuilt again after the war and expanded in 1984. The neighborhood holds its scars and its continuations in the same buildings.

Getting to Central from Shau Kei Wan was once famously difficult. A saying circulated: 英雄被困筲箕灣，不知何日到中環 — 'A hero is trapped in Shau Kei Wan, not knowing which day he will reach Central.' The distance isn't far, but congestion on the single road connecting the eastern end of Hong Kong Island to its western commercial core made the journey unpredictable enough to become a local proverb.

A tram line reached Shau Kei Wan from North Point in 1904 — a single track on what became the longest tram route in Hong Kong. One of the seven tram termini of the Hong Kong Tramways system sits at the junction of Shau Kei Wan Main Street East and Kam Wa Street. Some residents still ride the trams for tradition and their extraordinarily low fares, even now that the MTR connects Shau Kei Wan to Central in 18 minutes. The wholesale fish market at 37 Tam Kung Temple Road — the second-largest in Hong Kong after Aberdeen — still operates in the morning hours, as it has for decades.