Gjorwell had learned his craft from the French architect Louis Jean Desprez, who had designed both the royal pavilion and the abandoned castle ruins still visible nearby. For inspiration, the young architect turned to a villa he had designed himself at Drottningholm for the ballet-master Louis Gallodier. Construction moved swiftly. The foundation was laid in May 1802, and by year's end the building was roofed. Herman Edberg oversaw the work, commanding not just skilled craftsmen but infantry soldiers from the Sodermanlands regiment pressed into construction labor. Oak floors salvaged from Fredrikshov Palace and stone originally cut for Gustav III's dream castle found new purpose here. By late 1805, the interior was complete. The modest palace stood as a quiet rebuke to royal excess, a home built from the fragments of grander ambitions.

The palace passed through royal hands like a family heirloom. Crown Prince Oscar and Josephine used it as a summer retreat in the 1820s. In the 1860s, Prince August and Princess Therese moved in after renovations. Therese would remain for 54 years, her personality imprinting itself on every room until her death in 1914. Then silence fell. The palace sat empty until 1932, when Hereditary Prince Gustaf Adolf and Princess Sibylla breathed new life into its halls. Here their five children were born: Margaretha, Birgitta, Desiree, Christina, and finally Carl Gustaf, the future king. The palace echoed with children's laughter until tragedy struck again. On January 26, 1947, Gustaf Adolf died in a KLM DC-3 crash at Copenhagen Airport. The widowed Sibylla and her children withdrew from full-time residence, and the palace's role as a royal nursery ended.

In 1966, King Gustaf VI Adolf relinquished the palace to the government, transforming it into a guesthouse for distinguished foreign visitors. The arrangement seemed permanent, yet the palace would prove reluctant to leave royal hands entirely. Its use as a state residence proved sporadic at best, the rooms too intimate for formal diplomacy, too historic for practical modernization. For four decades, Haga Palace drifted in institutional limbo, neither fully royal nor wholly governmental, its nurseries converted to reception rooms that rarely received anyone.

On April 23, 2009, Prime Minister Fredrik Reinfeldt announced that Haga Palace would return to the royal family. The occasion was the engagement of Crown Princess Victoria to Daniel Westling, and the government offered the palace as a wedding gift. The couple married on June 19, 2010, and moved into their new home that November. The palace where the current king had been born now housed the future queen. The recycled columns, the salvaged floors, the stones meant for a murdered king's dream castle all found their purpose at last: supporting not ambition but family, providing not grandeur but shelter. In Haga Park, surrounded by the English gardens Gustav III had planted before his death, the modest villa built from fragments continues to hold Swedish royalty together.