Dalrymple's replacement was John Herbert, who arrived at Balambangan aboard the Britannia in December 1773 with soldiers, goods, and supplies from India. The settlement began trading opium, munitions, and fabrics with the Tausug and Maguindanaon peoples. But Herbert proved even more problematic than Dalrymple. He mismanaged the settlement and cultivated poor relations with the Tausug chiefs, whose hospitality the British presence depended upon. The consequences came swiftly. On 26 February 1775, Moro pirates attacked and destroyed the settlement. Herbert and a handful of survivors fled to Brunei. The Company's first venture at Balambangan had lasted barely fourteen months.

Balambangan did not stay empty. In the second half of the 18th century, the Dutch attempted to settle the island's western coast, operating from their base at Batavia. That effort also withered, and they withdrew by 1797. Then, in 1803, European power politics handed Balambangan back to Britain. The Treaty of Amiens of 1802 had returned the Moluccas to the Dutch, and Lord Wellesley, Governor-General of India, decided the Company needed a new strategic foothold in the region. He dispatched R. J. Farquhar, the British Resident at Amboina, to resettle Balambangan. Farquhar managed to re-establish the settlement by September 1803, though the journey was plagued by misfortune -- two ships were wrecked off neighboring Banggi Island en route from Malacca, and another was destroyed by fire in Balambangan's own harbor.

Farquhar departed for Penang in December 1803, leaving a commissioner in charge and placing Balambangan under Penang's jurisdiction. As lieutenant governor of Penang, Farquhar drew up plans to fortify the settlement. But history intervened once more: the resumption of war with Napoleon consumed British resources and attention. The court of directors vetoed the fortification plan, and in 1805 the island was abandoned for the second and final time. That same December, five members of the crew of the shipwrecked schooner Betsey escaped from Balambangan in a small boat after being attacked by eleven islanders. One sailor died of his injuries during the passage. The island that Dalrymple had imagined as a jewel of British commerce in the East was left to the sea, the jungle, and whoever chose to call it home on their own terms.

Balambangan sits just three kilometers west of Banggi Island, within the Tun Mustapha Marine Park -- at over 8,000 square kilometers, one of the largest marine protected areas in Malaysia. The colonial forts and trading posts are long gone, reclaimed by tropical vegetation. What remains is the geography that made the island attractive to Dalrymple in the first place: its position at the gateway between the South China Sea and the Sulu Sea, commanding the northern approaches to Borneo. The waters around it teem with coral reef life. The forests have regrown. Balambangan's story is one of failed ambition and abandoned empire, but the island itself has outlasted every plan that was ever imposed upon it.