The first castle here went up around 1100, a lowland fortress built by the Counts of Oldenburg to control the long-distance trade route running from Westphalia north to East Frisia. The keep is mentioned in a document of 1313 and was already in trouble by 1573, when the castle was officially declared dilapidated. Around 1400, under Count Dietrich, the buildings had been gathered inside a ring moat with an outer wall, turning the place into a proper moated castle. The waters of the Hunte filled the moat. The court that lived inside was small but dense, and the constant subsidence of foundations in soft ground meant something was always either being shored up or torn down.

Anthony Günther of Oldenburg, who reigned from 1603 to 1667, was the count who turned the castle into a palace. He hired the master builder Anton Reinhardt in 1607, who began the first masonry work that survives in today's building, and from 1609 to 1615 the project was directed by Andrea Spezza, a Ticinese architect from Arogno in Switzerland. The sculptor Ludwig Münstermann shaped the Renaissance facade. The model was Italian: a regular four-wing palazzo arranged around a courtyard, the kind of formal building a Venetian or Florentine nobleman would recognise. Then the Thirty Years' War turned northern Germany into a battlefield and Anthony Günther ran out of money. The project stopped, half-finished, and stayed half-finished for the rest of his life.

Anthony Günther died in 1667 without a legitimate heir. His lands fell to the elder line of the House of Oldenburg, the Danish royal family descended from Christian I, the Count of Oldenburg who became King of Denmark in 1448. For more than a century Oldenburg was governed in personal union with Denmark, and a Danish governor lived in Anthony Günther's unfinished palace. The Danes had little affection for the place. The last remains of the medieval castle had to be removed in the eighteenth century because the buildings were collapsing, and the moat that had defined the site for four hundred years was largely filled in at the same time. In 1744 the Danes added a plain chancellery wing for the state administration.

In 1773 the Holstein-Gottorf branch of the House of Oldenburg took over the newly created Duchy. From 1817 Peter I, Grand Duke of Oldenburg, modernised the interior under the neoclassical architect Heinrich Carl Slevogt and added a new wing for the court kitchen and library. The library wing burned down in 1913 and was rebuilt the same year in the same form. In 1894 the dilapidated Danish chancellery wing was demolished and replaced by a neo-Renaissance addition with ceiling paintings by the Bremen painter Arthur Fitger. By then the palace was the home of Grand Duke Friedrich August, the eleventh and final ruler of the line. He abdicated in November 1918 during the German Revolution and retired to his country seat at Rastede. The Free State of Oldenburg took the palace over in 1919-20 and turned it into the Landesmuseum Oldenburg, which it has been ever since.

The Landesmuseum holds the decorative arts and local history collections of the State Museum for Art and Cultural History, plus a respectable group of Old Master paintings. The Idyllenzimmer, the Idylls Room, displays a cycle of pastoral landscapes by Johann Heinrich Wilhelm Tischbein, the court painter who travelled to Italy with Goethe in 1786. The main festival hall, restored in its neo-Renaissance form, still functions for ceremonial events. Outside the west and north walls is the Schlossplatz, a busy public square. Across it, the Schlosshöfe shopping mall opened in 2011 in a deliberately contemporary architectural register. South of the palace are the Prinzenpalais and Augusteum, both part of the same museum complex. To the southwest the Elisabeth-Anna-Palais sits at the edge of the Schlossgarten.