With financial backing from Cincinnati banker Lewis Seasongood, Elsas built a new complex on the south side of the Georgia Railroad line, east of downtown Atlanta, on the former site of the Atlanta Rolling Mill. By 1881 the operation had become the Fulton Cotton Spinning Company. A bag factory followed in 1882. When the partnership with Isaac May dissolved, Elsas incorporated his half as the Fulton Bag and Cotton Mill Company in 1889. Growth was relentless. A second mill with more than 40,000 spindles opened on the Atlanta site in 1895. A third added 50,000 more by 1907. The company acquired bag plants in New Orleans and St. Louis during the 1890s, then opened mills in New York and Dallas in the early twentieth century. Plants in Minneapolis and Kansas City came during and after World War I. A Denver facility was added in 1945. Back in Atlanta, offices, picker buildings, warehouses, and the Jacob Elsas Clinic and Nursery filled out the sprawling campus.

Labor trouble shadowed the prosperity. A wage dispute triggered a two-day walkout in 1885. In 1897, white workers struck for five days to protest the hiring of 25 Black women. But the most significant conflict erupted in 1914, when workers began organizing with the United Textile Workers. Management pushed back, and the resulting strike brought demands for higher wages, a 54-hour work week, and limits on child labor. The confrontation drew national attention in March 1915 when the newly created U.S. Commission on Industrial Relations sent representatives to Atlanta to take testimony. Despite the spotlight, the strike collapsed in May 1915. The Elsas family maintained control, passing the company presidency through generations: Jacob to Oscar in 1914, Oscar to Benjamin in 1924, then to grandsons Norman, William, and Clarence through the 1940s and 1950s. Jacob Elsas himself was a civic force beyond the mill, helping found the Georgia Institute of Technology and supporting Grady Hospital, the Grand Opera House in Macon, and the Hebrew Orphan's Home before his death in 1931.

The postwar world changed what people put things in. Multiwall paper bags, canvas goods, and barrier materials replaced the old cotton products. In 1956, Eastern and Midwestern investors bought controlling interest. The nine bag manufacturing companies were sold off, and by 1960 the parent company had become Fulton Industries Inc. The Atlanta mill continued under Elsas family management until 1968, when Fulton Industries was sold to Allied Products Corp. The mill's last president, Meno Schoenbach, oversaw the final years of operation until 1977, when the Atlanta plant closed its doors for good. A century of cotton spinning, bag making, and industrial grit came to an end in the Cabbagetown neighborhood that the mill had built.

The buildings sat idle until 1997, when Aderhold Properties began converting the historic mill into 505 loft apartments called The Fulton Cotton Mill Lofts. In 1999, a major fire during construction gutted one of the buildings. The incident became an Atlanta television event when a crane operator was trapped at the top of his rig, unable to escape, until an Atlanta Fire Department firefighter dangled from a helicopter cable on live TV to rescue him. The shell survived, and the interior was rebuilt. Then, on March 14, 2008, an EF2 tornado ripped the roof off Building E, pancaking several floors below in a domino effect. Only 15 units had been sold in that phase; fortunately, the few owners present escaped unharmed. Aderhold also converted three rental buildings into condominiums marketed as The Stacks, named for the mill's still-standing smokestacks.

The mill complex has lived multiple lives on screen. Its derelict condition provided the gritty backdrop for the 1992 film Trespass, with broken support beams, collapsed floors, and water-damaged walls adding authentic atmosphere. The Fulton Bag and Cotton Mill appeared as a setting in the Academy Award-winning Driving Miss Daisy. More recently, the renovated lofts became familiar to millions as the home base of the Fab 5 in Netflix's 2018 reboot of Queer Eye, and the complex was featured in the 2019 film What Men Want and the reality series Love & Hip Hop Atlanta. Today the old smokestacks rise above a neighborhood of artists, young professionals, and longtime Cabbagetown residents, a place where the industrial bones of Jacob Elsas's vision still frame the daily life of modern Atlanta.