Congress appropriated $12,000 in June 1906 to build a permanent Yard Dispensary. The contractors stayed away — no outside bid came in — so Yard labor built the structure themselves. In December 1908, the wooden building opened on brick piers near the center of the Navy Yard. A basement was added later. The dispensary became a small hospital, with overflow patients still housed in tents. By 1917, the original 28 beds were nowhere near enough for the World War I personnel surge. The Naval Emergency Fund Act paid for an actual hospital. The Charleston Engineering & Constructing Co. broke ground June 1, 1917, and the new facility was commissioned just two months later, on July 31, despite a carpenters' strike. By September 1918, fourteen additional buildings raised the bed capacity to 1,000.

World War I ended, the patient load fell, and in 1922 the Bureau of Medicine and Surgery decided maintaining two parallel medical facilities at Charleston was not worth the cost. The emergency hospital was officially closed on December 21, 1922. Personnel transferred back to the older Yard Dispensary. Buildings from the emergency hospital were either moved or torn down for materials. For most of the 1920s and 1930s, the medical operation at the Navy Yard ran out of a frame structure with about 57 beds — adequate for routine peacetime work, modest by any other measure. The Great Depression brought Works Progress Administration money in September 1940, about a million dollars for a new 200-bed hospital that would actually be needed sooner than anyone expected.

Construction on the new hospital began in spring 1941, on the site of the old World War I emergency hospital. It was commissioned April 13, 1942, with 380 beds in permanent wards. By the end of the year, temporary wards had brought the total to 600. In September 1944, work began on a 260-by-42-foot recreation building. Inside, Quartermaster First Class Wilko H. Anderson — a civilian portrait painter before the war — covered the walls with life-size murals of Navy, Marine, and Red Cross figures around the seal of the Bureau of Medicine and Surgery. A 14-by-35-foot map of the world stretched across the south wall. Famous cartoonists who visited drew their characters directly on the Red Cross Service Room walls. The building also held a post office, barber shop, beauty shop, library, and a 542-seat movie auditorium that ran films twice each evening.

On February 14, 1970, the Navy broke ground on a new 500-bed hospital — a modern ten-story structure on Rivers Avenue in North Charleston. It dedicated on March 2, 1973, at a final cost exceeding $18.5 million. The new facility served roughly 73,000 eligible patrons, with two intensive care units, seven operating rooms, three delivery rooms, central oxygen and vacuum systems, and televisions in patient rooms. At its peak in the early 1990s, more than 1,200 personnel staffed Naval Hospital Charleston. The hospital was delivering over 1,300 babies a year, performing more than 3,000 surgeries, admitting 9,000 inpatients, and treating 365,000 beneficiaries — the high point of its institutional history.

In 1993, the Base Realignment and Closure Commission held hearings at the Gaillard Auditorium downtown. The decision was severe: most of the Navy presence in North Charleston would leave. By the end of fiscal year 1995, the active-duty population and family members had dropped from 77,000 to fewer than 38,000. The hospital downsized to 40 beds. In 1996, the Charleston Navy Base and Shipyard closed entirely. The hospital's emergency room and ICU were disestablished in February 1998. The last inpatient ward closed April 14, 1999. Operating rooms on the tenth deck shut down in October 2006. In January 2007, the institution renamed itself Naval Health Clinic Charleston, acknowledging that it was no longer a hospital but an ambulatory care facility. In 2010, operations consolidated at a new 188,000-square-foot joint VA-Navy clinic at the Naval Weapons Station in Goose Creek. The Rivers Avenue tower stood empty. The murals on the old recreation building, painted by a portrait painter who had set down his civilian brushes to serve as a Quartermaster, are part of why the historic district was preserved.