Douglas Lapraik made his fortune through shipbuilding and dock renting, and by the standards of mid-19th century Hong Kong, he made it spectacularly. He was among the wealthiest people in the colony, a Tai Pan — a term used for the most powerful foreign merchants — in the fullest sense of the word. The castle he built on Pok Fu Lam hill reflected that status: influenced by English Tudor architecture and Gothic styles, it originally featured an octagonal penthouse bedroom that faced directly toward the sea.

The location was deliberately chosen. Pok Fu Lam Road was developing as the main connection between Hong Kong's Western district and Aberdeen, where Lapraik's docks would sit. From his hillside perch, he could monitor his fleet entering and leaving the harbor. In 1866, Lapraik returned to Great Britain permanently, leaving the castle to his nephew, John Steward Lapraik.

In 1894, bubonic plague reached Hong Kong. The epidemic was devastating: thousands died, and roughly half the population — those who could leave — fled the city. Among those who remained were the members of the French Mission (the Paris Foreign Missions Society), who purchased Douglas Castle from John Douglas Steward, Lapraik's successor, when he could find no one else willing to buy. The building became a monastery, renamed Nazareth.

Under the leadership of Father Monnier, Nazareth was enlarged and transformed. A printing house was added on the northeastern side, and it became one of the busiest bible printing and translation facilities in Asia in the early 20th century. During World War I, the building served as a training base for the Royal Hong Kong Regiment. Then came the Japanese occupation of Hong Kong in December 1941: the Imperial Japanese Army confiscated Douglas Castle and used it as the headquarters of the kempeitai — the military police — and as housing for workers at the Japanese dock in Aberdeen. The printing press fell silent. The monastery ceased to be a monastery.

After the war ended in 1945, the French Mission recovered the building. The printing press resumed operations in 1948. But the context had changed irrevocably: the establishment of the People's Republic of China in 1949 closed the mainland to foreign missionaries, and Hong Kong as a base for mission work became increasingly untenable. By the early 1950s, the missionaries had largely departed. Nazareth closed.

The University of Hong Kong purchased the building from the French Mission on 4 December 1954, for HK$1,600,000. The chapel was converted into a dining hall; the crypt became a common room. The printing workshop was demolished and replaced with a car park. In 1956, the first residents arrived — about 52 students from Eliot Hall, Morrison Hall, and Lugard Hall — and University Hall began its life as a place of learning.

The early years had a formal quality that would seem extraordinary today. Hallmates wore green gowns to dinner. High Table Dinner was held every Monday. Residents who left the hall overnight had to report to the warden. The discipline was strict, and so, apparently, was the ambition: in the first 12 years of University Hall's existence, 8 of the 12 presidents of the Hong Kong University Students' Union were residents of the hall. The hockey team won the Malayan Cup in 1966 and 1968.

On 7 September 1995, the building was declared a monument under the Antiquities and Monuments Ordinance — one of Hong Kong's highest heritage designations.

Every long-lived institution accumulates folklore, and University Hall is no exception. Its residents — known as Castlers — are custodians of three treasures that define the hall's identity.

At the main entrance stairs stand three Père David's deer: two adults and one young. Students who touch the deer before graduating are said to be cursed never to graduate; the antidote is to touch the rock lions at another entrance. The bronze spiral staircase inside the southern tower is said to have been red originally — the paint wore off over decades of neglect — and legend holds that the staircase was taken to Japan during the occupation and returned afterward at the demand of the Hong Kong government. It is said to be one of only two bronze spiral staircases in Hong Kong.

The third treasure is not an object but a person: Ms Yuen So Moy, known as Sam So — literally "Auntie Three" — who worked as a cook at University Hall from its founding in 1956 until her retirement in 1998. She was a constant across decades of change, a mother figure to generations of students. The University of Hong Kong awarded her an honorary degree in 2009. A Castler, over the course of their time at the hall, tastes the hall blood she prepares three times: on becoming a resident, on graduation, and on marriage. It is a blessing, a ritual of belonging, and a thread connecting the present to the hall's beginnings.