Retiro began as a royal extension of the Hieronymite monastery of San Jeronimo el Real. Philip II enlarged the grounds in the 1560s, and in the 1620s his successor's chief minister, the Count-Duke of Olivares, transformed the area into an elaborate pleasure garden surrounding the newly built Buen Retiro Palace. The Italian engineer Cosimo Lotti, who had previously worked on Florence's Boboli Gardens, designed the layout around a series of water features: a great pond, a canal, and ornamental pools. During the Spanish Golden Age, plays were staged among the gardens for Philip IV and Charles II. The park was the center of Habsburg court life, a place of spectacle and escape at the eastern edge of a growing capital.

For over two centuries, Retiro remained closed to all but royalty and their guests. Philip V added a French-style parterre; Ferdinand VI hosted Italian operas; Charles III replaced the old walls with wrought-iron railings that let ordinary Madrilenos at least see what they were missing. Napoleon's troops destroyed much of the palace and gardens during the Peninsular War, storing powder kegs among the flower beds and building a citadel on the grounds. Queen Isabella II replanted and landscaped, but in 1868, the Glorious Revolution toppled her and the park finally became public property. After nearly 250 years as a royal playground, Retiro belonged to the city.

In the late 19th century, Madrid used the park as a venue for international exhibitions. The results still stand. The Palacio de Cristal, a glass-and-iron pavilion designed by Ricardo Velazquez Bosco and inspired by London's Crystal Palace, was built in 1887 to display flower species from the Philippines. It sits beside an artificial pond, its glass walls reflecting the surrounding trees, and now serves as an exhibition space for contemporary art. Nearby, the Velazquez Palace, built for a mining exhibition in 1884, also functions as a gallery. The park's Paseo de la Argentina is lined with stone statues of Spanish kings, originally sculpted for the Royal Palace between 1750 and 1753 but relocated here when they proved too heavy for the palace roofline.

In 2021, Retiro Park and the Paseo del Prado were jointly designated a UNESCO World Heritage Site, recognizing the cultural landscape that connects Madrid's great museum boulevard to its greatest park. On Sunday mornings from late May through early October, the Banda Sinfonica de Madrid gives free concerts from the park's bandstand. Rowing boats ply the Estanque del Retiro beneath the semicircular colonnade of the Monument to Alfonso XII, an early 20th-century construction topped with an equestrian statue of the king. Puppet shows, street performers, and fortune tellers cluster around the lake. The Rose Garden, inspired by the Bagatelle garden in Paris's Bois de Boulogne, blooms through spring and summer. In a corner of the park, the Forest of Remembrance commemorates the 191 victims of the 2004 Madrid train bombings with a grove of olive and cypress trees. An olive tree planted in 2023 is estimated to be 627 years old, meaning it was alive when Columbus sailed.