Kaunas sits where two of Lithuania's great rivers join: the Neris flowing in from Vilnius to the east, the Nemunas curving up from the south. Boats laden with grain, amber, fur, and wax could move between them. Anyone holding the confluence held the trade. The Lithuanian rulers built their first stone castle here in the middle of the fourteenth century, an outpost roughly a hundred kilometers downriver from their capital. The Teutonic Knights, crusader-soldiers headquartered in Marienburg, wanted the river system for themselves and the conversion of pagan Lithuania for their God. In 1361 the Grand Master Winrich von Kniprode ordered intelligence on the castle's wall thickness. The next spring he came with siege towers and, possibly, the earliest gunpowder weapons used in this part of Europe.

Kęstutis rebuilt almost immediately. In 1384 the Knights took the castle again, renamed it Marienwerder, and began their own reconstruction. Lithuanian forces sailed downstream from Vilnius the same year with cannon and trebuchets, and the castle changed hands once more. After 1398 the Teutonic Knights would never recover it. The Lithuanians built thicker—walls three to three and a half meters thick, ten meters tall—and added four towers. The southeastern was round, the southwestern square, both four stories high. By 1409, on the eve of the Battle of Grunwald that would break Teutonic power for good, six hundred soldiers garrisoned Kaunas Castle. After Grunwald, the castle's military function softened. Vytautas the Great stayed here often. Władysław II Jagiełło convened gatherings within its walls. In 1408 Kaunas received Magdeburg rights, and the castle became the administrative heart of a city granted the freedom to govern itself.

After Vytautas's death in 1430, the castle's prestige slowly faded into different uses. It became a prison for noble captives. Sheikh Ahmed, the last Khan of the Great Horde—a successor state of the Mongol Golden Horde—was held in its cells after Lithuanian forces captured him at the start of the sixteenth century. In 1549, King Sigismund Augustus gave the castle to his wife Barbara Radziwill as a wedding gift; her secret marriage to him scandalized the courts of Poland and Lithuania. A massive artillery bastion forty meters across was added in the sixteenth century, walls twelve meters tall, paired with a defensive trench. Then nature began its slow demolition. In 1611, the Neris flooded part of the castle. The mid-seventeenth century brought worse floods and a war with Sweden that left the place battered. By the eighteenth century it was a prison. Russian administrators eventually allowed houses to be built on its grounds, and the medieval stones disappeared beneath ordinary streets.

Rescue came slowly in the twentieth century. In 1930, Lithuanian authorities cleared the residential houses from inside the castle yard and began archaeological work. Serious preservation resumed in 1954. The round tower was repaired. The buried artillery bastion was excavated from beneath the strata of centuries and found, remarkably, in excellent condition. Decisions were made not to reconstruct the towers and walls to their full medieval height; what you see today is mostly foundations, with the round tower and a portion of curtain wall standing as tangible survivors. A 2010–2011 reconstruction added protective roofing and made the site accessible. The castle now houses a branch of the Kaunas City Museum. Roughly a third of the original stronghold remains.

From the ramparts you can still see why kings cared about this corner of Europe. The Neris and Nemunas meet directly below; the wide bend of the rivers spreads into the distance. The Romantic Polish poet Adam Mickiewicz set his 1828 poem Konrad Wallenrod amid the fourteenth-century Lithuanian-Teutonic wars partly inspired by this landscape. To stand at Kaunas Castle is to stand at one of Europe's older edges—the place where Christianity met its last sustained pagan resistance, and where the resistance held long enough that the Christianity it eventually accepted came on its own terms.