Governor Macquarie laid the foundation stone in July 1818, and after nearly three years of labour by convict stonemasons the Female Factory opened its doors in February 1821. It was designed by Francis Greenway, himself an emancipated convict, into elegant sandstone — but elegance was never the point. The factory was workhouse, prison, hospital, and marriage bureau in one. Women earned their way through a system of classes: First Class women could take wages, attend church, receive visitors, and marry their way out, with suitors given three days to choose a bride. Pragmatism, the records suggest, usually beat romance, and husbands sometimes returned wives they found disagreeable. Second and Third Class women got less food, less clothing, and harder treatment. When Governor Gipps later modernised the punishment cells, he ordered windows cut into the ground floor and the cells shrunk — changes so severe they horrified the British architects who had drawn the original plans.

Hundreds of children lived inside the factory walls alongside their mothers, and the colony's answer to convict motherhood was to separate the two. In 1844 a Roman Catholic Orphan School was moved next door — close enough that many of its children had mothers in the factory, a great number of them Irish. The reasoning was blunt: convict mothers were deemed unfit, their children better raised toward usefulness than left to 'poor choices.' The reality was bleak. A government report in 1855 found the place starved of food, utensils, clothing, and teaching, the children locked in at night and put to laundry and heavy labour instead of lessons. Inspectors noted the absence of the ordinary noise of children — no games, no spirit, just listless boys 'basking in the sun.' One visiting governor said the buildings looked like a place 'half-gaol, half-lunatic asylum.' The school limped on, chronically underfunded, until it closed in 1886.

From 1887 the ground took on its longest and darkest role as the Parramatta Girls Industrial School, holding girls labelled 'neglected,' 'wayward,' or 'uncontrollable' — including Aboriginal girls taken as part of the Stolen Generations. Many had absconded from violent homes, foster placements, or abuse, and were treated as delinquents for the very harm done to them. They were trained for domestic service and punished for the smallest infraction with beatings, isolation, and a cruelty called 'standing out,' where a girl held attention for hours. The testimony later given to a Senate inquiry is hard to read: a girl made to scrub concrete with a toothbrush in midwinter until her knees bled, staff who bashed and molested children. Riots flared again and again, as they had under the convict women, until a sustained campaign by the women's movement forced the school's closure in the mid-1970s. The full weight of what these girls carried into adulthood has only recently been believed.

What survives here is rare. Of the twelve female factories that once existed in colonial Australia, nine are gone entirely, which makes the convict-era ranges and walls at Parramatta a national rarity — among the only physical traces of how the colony chose to manage its women. After the factory closed it became the Parramatta Lunatic Asylum, later the Cumberland Hospital, and ran as a mental institution for over a century; one superintendent, Dr Richard Greenup, was stabbed to death by a patient in 1866 despite having pushed to ease the very confinements that defined the place. In 2017 the precinct was added to the Australian National Heritage List. The 2009 national Apology to the Forgotten Australians named the trauma carried out of institutions like this one. Historians now describe the site as a bridge — a shared space where a nation can finally witness experiences that thousands of women and children were once told no one would believe.