Construction began in the 1380s on a small island surrounded by water-filled moats, the perfect defensive position for controlling the strategic Aland archipelago. The castle first appears in historical records in 1388, when Queen Margaret I of Denmark received a charter transferring the estate of Bo Jonsson Grip, making her its new owner. From this isolated perch, whoever held Kastelholm controlled access to Stockholm, Turku, and the entire Gulf of Bothnia. The original structure featured a rectangular stone keep and residential wing, later merged into a single massive donjon known as Kurtornet. Kastelholm stands among Finland's five most architecturally significant medieval castles, alongside Hameenlinna, Olavinlinna, Raseborg, and Turku.

Gustav Vasa, before becoming Sweden's king, expanded Kastelholm significantly and hunted frequently in the surrounding forests. These woods were protected by law, reserved exclusively for the monarch and the castle's governor. By the sixteenth century, the castle had developed a shipyard employing around fifty shipwrights, turning the isolated fortress into a center of maritime industry. The castle's records from 1559 contain the first documented mention of Roma people in Finland, a historical footnote that speaks to the diverse traffic passing through this Baltic crossroads. In 1505, Danish naval commander Soren Norby captured the fortress, demonstrating that even this remote stronghold could not escape the constant struggle for Nordic dominance.

The civil war of 1599 brought artillery to Kastelholm's gates. Forces loyal to Charles IX of Sweden besieged and captured the castle during the War against Sigismund, leaving heavy damage that took until 1631 to repair. But the castle's administrative importance was already fading as Aland merged into the Abo and Bjorneborg County. In the 1660s, Kastelholm hosted the witch trials that swept through the region, its chambers witnessing the accusations and confessions that darkened Scandinavian history. Decades of neglect followed, and in 1745, fire gutted much of the structure. The castle briefly served as a prison before being abandoned in the 1770s. By the early nineteenth century, even the post office had relocated to nearby Bomarsund.

The twentieth century nearly finished what fire and neglect had begun. In the 1930s, local farmers used Kastelholm as a granary and even quarried its stones for building material. The medieval fortress that had imprisoned kings was being dismantled for barn foundations. Restoration finally came between 1982 and 1989, when archaeological excavations uncovered the castle's layered history. Today, the partially restored structure houses the Outdoor Museum Jan Karlsgarden, and visitors can examine excavated artifacts including early stove tiles in the castle's hall. The site is easily accessible from Mariehamn, drawing tourists who come to walk the same passages where Eric XIV once paced as a prisoner and where the fates of Nordic kingdoms were once decided.