There had been a castle here since the Middle Ages, a moated one, guarding a narrow neck of water in a landscape that is more lake than land. The von Bredow family acquired the lordship of Rheinsberg by marriage in 1464 and put up a Renaissance water castle on the spot in 1566. The Thirty Years' War wrecked it. After that the property changed hands the way small estates did: sold in 1618 to Kuno von Lochow, passed to Prince-Elector Frederick William when the Lochow line died out, handed on to a general named Franz du Hamel, sold again to Benjamin Chevenix de Beville. In March 1734 de Beville sold it to King Frederick William I of Prussia for 75,000 thalers. The king had a use for it. He gave it to the son he had spent years bullying.

Crown Prince Frederick moved into the southern wing in 1736 with his wife, Elisabeth Christine of Brunswick-Wolfenbuttel-Bevern. He set two architects to work, Johann Gottfried Kemmeter and his pupil Georg Wenzeslaus von Knobelsdorff, and between them they added an upper floor and pushed the east wing out by twenty-five metres. What emerged was the first full statement of the style later called Frederician Rococo, restrained and French-leaning rather than grand, and it became the working model for Sanssouci. Rheinsberg is where Frederick worked out what he wanted a building to look like. It is also, by his own repeated account, where he was happiest. He was twenty-eight when his father died in 1740 and the four years ended.

In 1744 Frederick handed Rheinsberg to his younger brother Henry, who moved in eight years later with his wife, Wilhelmina of Hesse-Kassel, and stayed for half a century. The king also built Henry a palace in Berlin between 1748 and 1753, the building that now houses Humboldt University, but Rheinsberg was where he actually lived and what he shaped. He extended the palace and the park. In 1774 he attached a theatre to the cavalier house, brought in travelling drama troupes and staged operas with a scratch orchestra. Then, in the early 1790s, he raised an obelisk on the far bank of the lake, in plain sight of the palace windows, honouring their brother Augustus William, who had died out of the king's favour, along with officers Henry believed the crown had slighted in the Silesian wars. He had their names cut into commemorative plaques. He built his own tomb too, while he was still alive: a pyramid, deliberately left broken, with a French inscription he composed himself. He was buried in it in 1802.

Rheinsberg's second life is literary. Theodor Fontane walked here for his Wanderungen durch die Mark Brandenburg and gave the palace a chapter of its own. Then in 1912 Kurt Tucholsky published Rheinsberg: Ein Bilderbuch fur Verliebte, a slim and very funny book about a young couple stealing a few days away together, and turned the place into a destination for exactly that. Readers arrived looking for the lake, the park and the feeling of having escaped for a weekend, and they have been arriving ever since. The palace museum now shares the building with a museum devoted to Tucholsky, which makes for an odd symmetry: the writer who made Rheinsberg famous for lovers had his books burned in 1933, lost his German citizenship the same year, and died in Swedish exile two years later.

The Hohenzollerns owned Rheinsberg until 1945, when the property was expropriated. What followed was less dramatic than demolition and rather more surprising: the palace spent the East German decades as a sanatorium for diabetics, named after a Social Unity Party politician. Since 1991 the former Cavalier House has held the Federal and State Music Academy, which runs the theatre Prince Henry built, and since the same year the theatre, the courtyard and the open-air Heckentheater have hosted an international opera festival every summer. The buildings and gardens now belong to the foundation that looks after Prussia's palaces. After a long restoration you can walk the rooms Frederick redesigned, which is as close as anyone gets to the four years he kept describing, for the next forty-six, as the best of his life.