Long before Screamers or saxophonists, Inishfree Upper was reportedly under the control of Niall of the Nine Hostages, the late 4th or early 5th century Irish chieftain who would become High King of Ireland and the legendary founder of the Ui Neill dynasty. The line of descent he established controlled enormous swathes of northern and central Ireland for centuries. In the late medieval period, Inishfree was held by his descendants in the line that became the O'Donnell clan, including Red Hugh O'Donnell, who led the last stand of the Ulster chieftains against the English in the Nine Years' War at the close of the sixteenth century. The island was a small possession in a vast Gaelic territory. After the Flight of the Earls in 1607 ended Gaelic rule, Inishfree slipped, like much of the Donegal coast, into the marginal economy that would persist for centuries.

In the early twentieth century, thirty-six families lived on Inishfree Upper. The island had a school. Two notable Gaeltacht writers taught there at different times: Séamus Ó Grianna, the prolific Donegal Gaeltacht novelist whose work appeared under the pen name Máire, and Peadar O'Donnell, the socialist republican writer and activist who would become one of the most important Irish political voices of the twentieth century. That two such figures both passed through the schoolroom of a tiny offshore island says something about the role of these places in modern Irish cultural history. The Gaeltacht islands were not just places where Irish was spoken. They were places where Irish was written, taught, and made into modern literature. The island's residents would mostly emigrate or move to the mainland in the decades that followed.

In September 1978, the Argentine pianist and composer Guillermo Cazenave was living on the island. He recorded an album there called Inishfree. The choice of a small Donegal island as a residence and recording studio for an Argentine musician fits a pattern that has held on this coast for over a century. Artists, writers, musicians, people looking for distance from the world, have found their way to these islands. Sir Arnold Bax at Glencolmcille, Robert Graves at Innisfree (different island, different spelling, in Yeats's poem), the Atlantis commune at Inishfree Upper. The island is a particular kind of place to come to. Cazenave came, recorded, left. The album exists, an Argentine pianist's response to a Donegal island.

The Atlantis commune was founded at Burtonport in 1974 by a group practicing primal therapy, the school of psychotherapy that holds repressed childhood pain can be released through unrestricted vocal expression. They came to be called The Screamers. The commune was led by Jenny James, a charismatic and divisive figure. The community moved to Inishfree Upper in 1980 to live more isolated. They occupied the cottages of the long-departed island families. They grew vegetables, raised animals, screamed when they needed to. The relationship with the surrounding Donegal community was complicated. In 1989, the commune relocated to Icononzo in Colombia. There, in 1998 and later, members of the community were killed by FARC guerillas. The story made international news, but its roots reached back to a small island in Donegal, a 1970s alternative therapy movement, and a particular set of choices about how to live.

Barry Pilcher, an English saxophonist from Essex, moved to Inishfree Upper in 1993. He became the last permanent resident. Because bad weather on this coast could prevent ballot boxes from being collected, Pilcher and the residents of nearby small islands were granted the right to vote a day early. In 2009, he and his island neighbours were the first people in Ireland to vote on the Treaty of Lisbon. The polling station was a whitewashed cottage; the photograph of it became briefly famous. Pilcher returned to Essex in 2013, ending nearly two decades as the island's last full-time inhabitant. Inishfree Lower, the smaller of the two islands, was home to one family until the 1960s or 1970s. Both islands are now uninhabited year-round, though some cottages serve as holiday properties. The island's role as a refuge for outsiders, lasted a long time, but did not last forever.