The site at Victoria Gap was first used in 1888, the same year the Peak Tram began operations, as a rest place and workshop for the British engineers who built the tram line. In 1901, the government took over the site and constructed a purpose-built chair shelter. When completed in 1902, the building was divided into two sections: one for public chairs and one for private. The walls were built from local blue stone with granite dressings, the floors were cement concrete, and the roofs were tiled. A space in front was left clear for chairs to stand during fine weather. It was built, as the report noted, for people who worked in the open on an exposed hillside — not for comfort, but for function and durability. The stone it was built from may have come from the nearby construction of Mountain Lodge, the Governor's residence on the Peak, which has since been demolished.

The building opened to the public in 1923, when sedan chair carriers were permitted to serve tourists at the Peak. It served that role until the Japanese occupation of Hong Kong between 1941 and 1945, when it was believed to have been used by the Japanese army as a police station. After liberation, the building's next transformation came swiftly: in 1947, a government official suggested converting it into an open-air café serving light refreshments. The suggestion was acted on. Arches were fitted with wood and glass doors; wooden flooring was laid over the original concrete. As a Peak Tram official wrote in 1977, summarising the changes since 1947: "Other than these minor modifications, the building stands as it was originally built."

By 1973, the Peak area had grown popular enough that a proposal was floated to demolish The Peak Lookout building and replace it with a car park. The resulting public outcry was decisive. A petition forced the plan to be scrapped, and in 1981 the Antiquities and Monuments Office awarded the building Grade II Historic Building status — defined as a building of special merit where efforts should be made to selectively preserve it. That designation protected it from further demolition proposals. In 1989, when the café's rental contract expired, a company called Freedragon Ltd. won the government bid to continue running it. The café became the restaurant it is today.

The building looks, at first glance, like something transplanted from the English countryside: a single-storey structure with a red, pitched roof, stone walls, arched windows, black and white half-timbering on the gable, and a visible chimney stack. A boundary wall and garden complete the impression. Inside, the restaurant is dimly lit and hung with old photographs of life on the Peak from generations past. The Peak Lookout is one of the few surviving examples of Arts and Crafts architecture in Hong Kong — a style associated with late-Victorian and Edwardian Britain that valued craftsmanship, vernacular materials, and an aesthetic rooted in the pre-industrial past. On a hillside at 396 metres, surrounded by subtropical forest, it is an unlikely but genuine example of the form.

From the open terrace, the view sweeps south and west: Aberdeen below, Pok Fu Lam Country Park stretching across the hillside, and beyond both, the South China Sea. On clear days, outlying islands break the horizon. This is the view that sedan chair bearers saw from their rest shelter in 1902, that Japanese forces saw from what they turned into a police post in the 1940s, and that diners see today over lunch. The Peak Lookout also operates a location at Terminal 1 of Hong Kong International Airport, which opened in November 2012 — but it is the hillside original, at Victoria Gap, that carries the weight of the name.