The Nemunas was Lithuania's commercial spine, and the bluff at Panemunė controlled a strategic stretch of it. The Teutonic Order built the first castle here in 1313, naming it Christmemel. They held it for less than a year. In 1314, Grand Duke Vytenis of Lithuania besieged the place for seventeen days and captured it, turning what had been a Crusader outpost into a personal residence of the Lithuanian rulers. After the Teutonic Order's catastrophic defeat at the Battle of Grunwald in 1410, the castle's military function faded. Stone walls became administrative offices, then a noble manor, then ruins. The river kept flowing past, indifferent to who held the high ground.

King Sigismund the Old granted the lands to the Stankevicius-Bilevicius family of Samogitia in 1535. They built a wooden manor and never quite finished what they started. When Stanislovas Stankevicius-Bilevicius inherited the estate in 1597, he sold it almost immediately to Eperjes, the Hungarian merchant whose name still attaches to the place. Eperjes commissioned Peter Nonhaardt, a Dutch architect already working on Vilnius Lower Castle, to design a proper residence. Construction ran from 1604 to 1610. The result blended Renaissance proportions with late Gothic structural details, two two-story wings joined by defensive walls and corner towers. It was meant to last centuries. It has.

By the close of the seventeenth century, the Eperjes family was selling off pieces of the estate to stay solvent. By 1753, an inventory recorded burnt-out furnaces, dried ponds, and only one of four mills still standing. The property changed hands repeatedly. In 1929, an emigrant priest named Antanas Petraitis bought the castle from his home in the United States and donated it to a Salesian order; after his death it passed to the Lithuanian Ministry of Education over unpaid debts. World War II interrupted any restoration. Then came Soviet occupation, and the Red Army did what Soviet troops did to old aristocratic buildings: vandalized the interior, broke what was breakable. A 1955 conservation project, prepared by Stasys Pinkaus, identified the priority work needed to save the structure from total collapse.

After Lithuania restored its independence in 1990, the castle was given to the Vilnius Academy of Arts. Restoration continued, but the choice of stewardship was unusual: rather than freezing the building as a museum tableau, the academy turned it into working studio space. Today the rooms display work by current art students alongside historical exhibitions. The grounds house a small museum. A few of the chambers operate as guest accommodation, which means visitors can sleep inside the walls Eperjes raised in 1610 and wake up to the same view of the Nemunas valley that the Hungarian merchant must have considered when he decided to spend a fortune here.

The aerial view tells the story most clearly: a compact rectangular plan with corner towers, the river curving below, parkland and ponds surrounding the brick walls. The Renaissance bones are still visible under centuries of patching. Compared to the famous reconstructed castles of Lithuania, Trakai's lakebound fortress or the Palace of the Grand Dukes in Vilnius, Panemunė is the unsung survivor: less spectacular, less photographed, but more intact, more original, more honest. The Litas commemorative coin issued for it captured what most travelers eventually realize on arrival. This is what a Lithuanian castle actually looked like, before the wars and the centuries got to most of the others.