Inscriptions on the facade of the Great Mosque record the founding: the medina was built between 849 and 851, following orders from Abu Abbass Muhammad, the Aghlabid emir of Kairouan. The cadi Ali Ibn Salem oversaw the construction at Sfax. The Aghlabids had already built a fortified kasbah on the coast as part of their network of coastal defenses, and the city grew around it. But religious politics complicated the process. The Aghlabids promoted the Ismaili rite, while the local population clung to the Maliki doctrine, supported by the scholar Abu Ishaq al-Jabanyani. The medina that emerged reflected this tension: a city built by imperial decree but shaped by local conviction, its great mosque serving as the point of contact between the two.

What distinguishes the medina of Sfax from other Tunisian medinas is the particularity of its domestic architecture. Houses built between the seventeenth and eighteenth centuries, the period of the city's greatest economic expansion, follow a distinctive pattern. The patio is central, its decoration and dimensions reflecting the family's social status. Rooms open onto this courtyard, always rectangular, with high ceilings that provide natural ventilation in the hot climate. In wealthier homes, rooms have a T-shape: a central reception space called a kbou for welcoming guests, flanked by two lateral cells called maksoura for children. As the population grew in the eighteenth century, upper floors began to appear, sometimes as extensions of the ground floor, sometimes as completely separate dwellings with their own street entrances. The walls of the medina run 2,750 meters in length, with 34 bastions and heights varying between seven and eleven meters.

The souks are organized by specialty north of the great mosque, creating the economic center of the medina. This is traditional Islamic urban planning: sacred space at the core, commerce radiating outward, residential quarters beyond. Unlike many Tunisian medinas, Sfax has very few hammams, owing to its dry climate. The historian Mahmoud Megdiche records only four: Hammam El Sultan, Hammam El Mseddi, Hammam El West, and Hammam Ibn Neji. These served social as well as hygienic functions, hosting celebrations for marriages and circumcisions. Today only Hammam El Sultan survives, and it stands in poor condition. Several historic houses have been repurposed: Dar Jallouli became a museum, others became hotels, coffee shops, or artisan workshops. In 2012, the Tunisian government submitted a nomination to add the medina to the UNESCO World Heritage List.

The medina's walls have absorbed more than a millennium of upheaval. Under the Zirids in the tenth century, Sfax experienced an architectural renaissance even as the wider region descended into chaos. The city endured its first invasion under Ibn Melil from 1067 to 1099, supported by Hilali and Banu Sulaym Arab tribes. The Hafsids ruled from the thirteenth century until the Ottoman arrival. French bombardment in 1881 damaged the great mosque's minaret, and the colonial administration spent the next seven decades reshaping the city around the medina without destroying it. Seven national heritage monuments within the walls have official protection, including the great mosque, the walls themselves, and the Sidi Ilyes and Sidi Omar Kammoun minarets. Each newly built gate was traditionally given a temporary name, Bab Jedid, meaning New Door, until a permanent name was chosen. The practice captures something essential about the medina: it has always been changing, always adding, always absorbing the new while remembering the old.