Aboriginal people first came to Ulgundahi around 1880, retreating to the island as introduced diseases swept through and European farms swallowed the country they had always known. What began as refuge hardened into policy. In 1904 the reserve was gazetted, and the Protection Board began moving families in from Southgate, Ashby, Lawrence and Ulmarra. In 1908 it relocated families from Ashby, and the population climbed toward sixty. In 1909, people from Harwood Island fled here after a European farmer burned their camp of more than forty bark dwellings to the ground. Each family was allotted three or four acres for maize, cane and vegetables. The message was plain: become industrious, become useful to the settlers who had taken everything else.

The control was total in the way only bureaucratic control can be. Men cut cane and carried it to the Harwood mill; families tended peaches, oranges, mangoes, passionfruit and grapes, and prepared vegetables for the markets. But the money never reached the people who earned it. Every shilling from cane or produce went back to the Protection Board, and in 1911 the Board reported with satisfaction that only one resident still drew rations. A European manager came each day by boat from Ashby to run the island and teach the school, where children learned little beyond what would fit them for labour. Boys were trained for farm work, girls for domestic service in white households. Families were never formally compelled to stay, yet a hostile society outside and a manager who could reassign any absent family's home made leaving its own kind of risk.

Inside those constraints, the Yaegl made a life that was theirs. In 1910 the residents built a small church, later furnished when John Cameron paid the balance owed, and an organ was carried in for Mrs Blakeney to play. Cardy Craig and his wife Sarah Blakey became, in 1909, the first Aboriginal people to own a home and land here. Families slipped off the island for weeks at a time to camps they loved: Murrayville on the river's north arm, rich with the wood worms they called cobra; Yamba on the coast for whole summers of fishing and gathering. This is country threaded with story. The Clarence itself, in Yaegl tradition, was shaped by Dirrangun, a powerful old woman of the creation time, and a great fig tree still standing at the island's western end is understood to hold spirits and must never be disturbed.

The river that defined the island also doomed the settlement. Flooding was relentless. The 1921 flood was a grim ordeal, families left to fend for themselves for up to three months; more floods followed in 1928 and through the 1940s and 1950s. In 1951 the government closed the school, and children rowed to Maclean for their lessons. In 1956 the Hillcrest reserve was created on the edge of Maclean, and one by one families moved to drier ground. The Protection Board manager retired in 1958 and was never replaced. By 1962 the last families had left for Hillcrest. Two skeletal corrugated-iron structures still stand among the vines, one of them said to be the first house ever built on the island.

The story did not end in departure. The island is owned today by the Yaegl Local Aboriginal Land Council, and the community still farms it, runs educational tours, and tends the country their ancestors held. In November 1996 Yaegl elders, among them Joyce Clague and Della Walker, lodged a native title claim that became the oldest such case in the Federal Court of Australia. It took nineteen years. On 25 June 2015, at an extraordinary hearing in Yamba, Justice Jayne Jagot delivered a consent determination recognising the Yaegl people's rights over land and the estuaries of the Clarence River from Harwood to the river mouth. Two years later, in August 2017, a second Federal Court determination made the Yaegl the first people in New South Wales to have ocean sea rights recognised. The island was listed on the State Heritage Register in 2004 not as a ruin but as proof. For the Yaegl, Ulgundahi stands for the injustices endured here and for the harder fact that they were not broken by them.