The station takes its name from Elsey Creek, which was named for Joseph Ravenscroft Elsey — the surgeon who travelled with the Augustus Charles Gregory expedition along the Roper River in the 1850s. Abraham Wallace took up the lease in 1879 and embarked on one of the great cattle drives of Australian history. Starting from his property at Sturts Meadow in outback New South Wales in January 1880, he pushed north to Longreach — 750 miles away — where he bought 2,728 head of cattle, then continued another 1,250 miles north, arriving at Elsey in July 1881 after covering roughly 2,000 miles total. He left the next day to return south. The first homestead was built at Warlock Ponds and later moved to Red Lily Lagoon. The country Wallace had claimed is 5,334 square kilometres of flood plains, black-soil country, red sandy ridges, and tropical savanna threaded by the Roper River and its tributaries.

Elsey's early decades were marked by the brutal patterns common to colonial pastoral expansion. On 15 July 1882, head stockman Duncan Campbell was fatally speared. The reprisals were devastating: what became known as the Red Lily Lagoon Massacre killed approximately 20 Mangarrayi people — men and women whose connection to this country predated the station by millennia. Wallace himself never truly benefited from his extraordinary effort. He had been managing the property remotely from Adelaide when, on 27 April 1884, he was found dead, his throat cut; the coroner found his mind had been unhinged by a buggy accident six weeks prior. In January 1895, an Aboriginal man named Moolooloorun was hanged at Crescent Lagoon on the station. The land accumulated grief alongside cattle.

When Jeannie Gunn returned to Melbourne after Aeneas's death, she transformed her brief, intense experience of station life into We of the Never Never — a portrait of characters, landscape, and the peculiar society of the outback that became an Australian classic. A 1982 film adaptation brought a new generation to the story. The original homestead had migrated by then: after Aeneas's death it was moved from Red Lily Lagoon to McMinn's Bar; a replica built for the film now stands at the Homestead Tourist Park in nearby Mataranka. In 1946, the film crew for The Overlanders spent a month at the Roper River camp, shooting a river crossing sequence. Elsey's landscape had become, in a sense, part of the national imagination.

The station changed hands multiple times across the twentieth century, reaching an advertised herd of 26,000 cattle at its 1951 auction. But the most significant transfer came quietly, in a ceremony, in February 2000. Howard government representative John Herron handed the title deeds to the Mangarrayi people, whose native title claim — nearly nine years in resolution — had finally succeeded. The Elsey Land Claim, granted in 1997, recognised the Mangarrayi as the traditional owners of a place that had carried their name in its creeks long before any European surveyor gave it to a surgeon. The property is now owned by the Mangarrayi Aboriginal Land Trust. About 7,000 cattle still graze its pastures.