The mission takes its name from Carlo Borromeo, the sixteenth-century Archbishop of Milan. Serra first established the mission in Monterey on June 3, 1770, near the native village of Tamo, making it only the second Franciscan mission in Upper California. But relations with the Spanish military commander were strained, and the Carmel River valley offered better farming conditions. In May 1771, Spain's viceroy approved Serra's petition to relocate. By August 1, 1771, the mission stood in its new location. The first mass was celebrated on August 24, and Serra took up residence in the newly constructed buildings on December 24. The mission was within a short distance of the Rumsen Ohlone villages of Tucutnut and Achasta.

After Serra's death, Father Fermin Lasuén replaced the original adobe structure with one built of stone quarried from the nearby Santa Lucia Mountains. The mission served as headquarters for all Alta California missions until 1833, when the Mexican government secularized the mission buildings and lands. The native population had already declined dramatically. Deaths exceeded births, and the population at Mission San Carlos peaked in 1795. By the mid-nineteenth century, the structures had fallen into ruins. In 1884, workers rebuilt the chapel roof, saving it from total destruction. Two years later, ownership transferred from the Franciscans to the Diocese of Monterey, and Carmel Mission has been an active parish ever since.

Beginning in 1931, Harry Downie began restoring the mission and worked continuously on the project for the next fifty years. He rebuilt the former soldiers' quarters on the east side of the quadrangle in 1941. In 1943, he began restoring a building on the south side from nothing but eroded adobe foundations and ruined walls. This building had been the monjero, a locked dormitory where girls were separated from their families at age eight. Downie also restored the original padre's kitchen and blacksmith shop, now used as a chapel. His work extended beyond Carmel: he consulted on restorations at San Luis Obispo, San Juan Bautista, and San Buenaventura missions, and helped reconstruct Mission Soledad.

The mission is the only Spanish mission in California that retains its original bell and bell tower. The bell, nicknamed Ave Maria, was made in Mexico City in 1807 and placed at the mission in 1820. When secularization came in 1834, local Native Americans removed the bell and kept it safe. It was eventually lost, then relocated during the twentieth-century restoration. Workers reinstalled it in the mission bell tower in 1925. The mission also contains California's first library, established during the mission period. In the nave hang original Spanish Colonial oil paintings conserved between 2000 and 2007, including the Deposition, a painting commissioned by Father Serra before his death that now hangs near his tomb in the apse.

Today the mission operates as an active parish church of the Roman Catholic Diocese of Monterey. Visitors can tour the Convento Museum, which holds the cell where Serra lived and died, along with some of his vestments displayed in a special chapel room. The Junipero Serra School operated on the grounds from 1943 until closing in 2020. Restoration continues: in 2016, the Carmel Mission Foundation funded a $2.2 million restoration of the central courtyard. In late 2020, the foundation began a $4 million project to restore and seismically strengthen the Downie Museum in time for the 250th anniversary of the mission's founding. The foundation also conserved the Our Lady of Bethlehem statue, believed to be one of the oldest statues in California.