Haviland belonged to a generation of reformers who believed prison design could rehabilitate rather than simply punish. His jails emphasized individual cells, natural light, and separation from the corrupting influence of other prisoners — the 'separate system' that Eastern State Penitentiary embodied in its most extreme form. The Essex County Jail was built in the Greek Revival style from brick and local brownstone, with a two-story administrative building at the front and a three-story sandstone cell block behind it, measuring roughly 75 by 100 feet, with tiers of cells running down the center. The building sat at the corner of Newark and New Streets at what was then the city's edge, adjacent to the Morris Canal — close enough to smell the water and hear the bargemen working the locks. Behind the administrative building, the canal formed the back of the property line.

The 1837 structure quickly proved insufficient. In 1890, and again in 1893, 1903, and 1907, Haviland's original building was expanded with multiple additions, pushing the cell count past 300. Each addition brought the jail further from its reformist origins and closer to the realities of urban incarceration: more bodies than the space was designed to hold, infrastructure straining under the weight of a growing city's criminal justice system. The women's wing, built in 1895, expanded the complex's reach. Running water and toilet facilities in each cell were later additions that updated the facility for a new era. By the mid-20th century, the complex consisted of about 20 structures spanning a century of American prison architecture — from Haviland's Greek Revival original to 1930s additions that reflected a very different institutional aesthetic.

In the summer of 1967, Newark burned. The rebellion that began on July 12 — sparked by the arrest and beating of a Black cab driver named John Smith — lasted five days and left 26 people dead, hundreds injured, and more than 1,000 arrested. Many of those arrested were held at the Essex County Jail, which was severely overcrowded during the crisis; records suggest the inmate population approached 500 when the jail's official cell capacity was around 300. The 1967 Newark Riots are part of why this jail was listed on the National Register of Historic Places in 1991 — the complex is understood not merely as an architectural artifact but as a site of social history, connected to one of the defining urban upheavals of the civil rights era. The following decade saw the jail decommissioned; a new facility was built in 1971, and Essex County's oldest public building was abandoned.

The jail has been vacant since 1971. In 1991, Spike Lee's crew used it as a filming location for Malcolm X — its aging walls and atmospheric decay providing an authentic period backdrop. In 2003, a fire devastated the Women's Wing, reducing it to rubble. The Warden's House, dating from 1837, has partially collapsed. Urban explorers photograph the decaying cell blocks; ruins photographers catalog the spreading damage. Plans to demolish it for a science park were rejected by Newark's landmarks committee in 2010. In 2017, Rutgers Law School applied to de-register the site from historic lists, which would have removed legal barriers to demolition. The application stirred opposition. Columbia University architecture students proposed eleven adaptive reuse schemes in 2018. For now, the jail decays, its future unresolved, its 1837 brownstone walls outlasting everyone's plans for them.