The first castle was strictly functional. A square fortress with a moat, three gates, and an inner courtyard, set on the bank of the Põltsamaa River in eastern Estonia. The Livonian Order — the German military-religious organization that had spent decades subduing the local Estonian and Livonian peoples — needed strongpoints from which to administer their conquests. In the 14th and 15th centuries the walls were raised again and again as artillery improved, and a convent hall and corner watchtower were added. By the 16th century, cannon towers protected both the southern and northern gates. The square box was now a serious fortification.

The Livonian War tore the medieval order apart. For eight years, between 1570 and 1578, Põltsamaa served as the official residence of Duke Magnus of Holstein — a Danish prince who hoped to carve out a Kingdom of Livonia for himself with the backing of Tsar Ivan the Terrible of Russia. Magnus was crowned king in Moscow in 1570 with elaborate ceremony, then sent west to make his claim real. From the rebuilt Põltsamaa he tried to govern lands that did not really belong to him, depending on Russian armies that came and went on Ivan's mood. The kingdom collapsed within a decade. Magnus eventually defected from his Russian patrons, lost Polish support too, and died in 1583 in straitened circumstances. The Polish troops who had occupied Põltsamaa during these years moved on. The castle waited for new owners.

In 1623 Sweden's King Gustavus Adolphus gave the estate to his Field Marshal Herman Wrangel as a personal gift. Wrangel had no interest in a medieval fortress. He installed fireplaces and tiled stoves, enlarged the windows and doors, added an extra floor, and converted the southern cannon tower into a parish church that still stands today. The courtyard became a household yard. The Great Northern War undid much of this — the castle burned and the interiors were destroyed — but the Russian reformer Heinrich von Fick took it over after 1721, courtesy of Peter the Great. By 1750 it had passed to Woldemar Johann von Lauw, who set out to do what Wrangel had only begun. Between 1772 and 1773 Lauw commissioned a complete rococo redecoration of the interiors, hiring the master stucco artist Johann Michael Graff — the same craftsman whose work survives at Rundāle Palace in Latvia. What Graff produced at Põltsamaa was, by general agreement, the most artistically accomplished rococo interior in Estonia.

The castle eventually passed to the Russian princely Gagarin family and stayed in their hands until the Estonian land reform of 1919. Then, on a single day in 1941 during the German invasion of the Soviet Union, an air raid almost completely destroyed it. The rococo interiors that had survived everything for nearly two centuries — the painted plaster, the gilded cornices, the carved doors — were gone in minutes. Today only ruins remain of the main building. The southern church Wrangel built from a cannon tower has been restored. Some of the annexes have been rebuilt as a small museum. Pre-war photographs preserved on Europeana show what the rococo halls looked like before the bombs found them. The drone footage shows what is left now: pale stone walls along a green river bend, with a white church beside them, and the river running on indifferent to all of it.