The monastery came first. In 1091, a count named Huno and his wife Willna founded a Benedictine house at Rastede in honour of the Virgin Mary, and their son Friedrich finished the building. The monastery church was consecrated five years later. In the twelfth century, the position of monastery bailiff became hereditary in a small noble family called the Egilmaren, named for their founding ancestor Egilmar I. Those Egilmaren are the people we now call the House of Oldenburg. Rastede became their house monastery, a private foundation tied to the family's spiritual and political identity, and so it remained for four hundred years, until the Reformation stripped its religious purpose away in the sixteenth century.

Count Anthony Günther of Oldenburg, who reigned during the chaos of the Thirty Years' War, built large new stables at Rastede in 1612. The former monastery was demolished in 1643 and replaced by a hunting lodge that doubled as a summer residence. When Anthony Günther died in 1667 without a legitimate heir, his lands passed to the Danish royal family, also descendants of the House of Oldenburg through Christian I, the Count who was elected King of Denmark in 1448. Denmark governed in personal union for more than a century and barely visited. The Danish king Frederick IV eventually parked the displaced Princess Sophie Eleonore there because she had nowhere else to go, and even she could not get enough money to repair the building she lived in.

In 1750 the Danish governor sold the palace to a judge named Christoph Römer, who hired the Dutch architect Cornelis Redelykheid to rebuild the place. Redelykheid worked from a Dutch model and produced a three-wing Baroque palace with a French formal garden out front. Twenty-seven years later, in 1777, the future Grand Duke Peter I of Oldenburg bought the estate back into the family. Between 1780 and 1791 he had the house redesigned in the more austere Neoclassical fashion of his moment, and in 1784 he appointed Carl Ferdinand Bosse as his garden architect. It was Bosse who first brought the rhododendron to the Ammerland district. The flower took to the local soil and is now the symbol of the entire region around Oldenburg.

By 1894, Rastede had been remodelled again, the north wing redone under Carl Heinrich Slevogt and Otto Lasius, the attic raised, the hall covered in stucco by the sculptor Eduard Demitrius Högl. Across the country road the Erbprinzenpalais, the Lodge of the Hereditary Prince, was rebuilt in the historicist style for Augustus's son. Then in November 1918, the Grand Duchy ended. Friedrich August II, the last reigning Grand Duke of Oldenburg, abdicated during the German Revolution and moved from his palace in the city out to Rastede. His granddaughter, Duchess Sophia Charlotte, lived in the Erbprinzenpalais after her divorce from Prince Eitel Friedrich of Prussia. The current Duke still lives in the main house. The Erbprinzenpalais, since restoration in the 1980s, hosts concerts and events open to the public.

The palace itself is closed to visitors, a private home for a family whose ancestors first laid the stones nine hundred years ago. The park is another matter entirely. Bosse's Schlosspark stretches around the building in the English landscape style he learned in the late eighteenth century, with rhododendrons that are now mature giants, paths threading between mature oaks and beeches, the Hirschtor or stag gate marking the formal entrance. The park is free and open all year. So is the Erbprinzenpalais across the road, with its own English landscape garden laid out in 1822 for Hereditary Prince Augustus. The village of Rastede, twelve kilometres north of Oldenburg, has the practical infrastructure of a small German market town. The palace is the reason people get off the train.