In 1228, Albert of Riga, the formidable archbishop who had hammered Christian Livonia out of the eastern Baltic, formed a new diocese covering western Estonia and the islands of Saaremaa and Hiiumaa. He named Gottfried, an abbot from the Cistercian monastery of Dunamunde, as bishop. The bishopric was created as a state of the Holy Roman Empire on 1 October 1228 by Henry, King of the Romans, and the boundaries were fixed by the papal legate Wilhelm of Modena in 1234. The Bishopric of Osel-Wiek then proceeded to move three times. The first residence at Lihula was abandoned to avoid conflict with the powerful Livonian Brothers of the Sword. The second at Perona (modern Parnu) was burned by Lithuanian raiders ten years later. The third site, finally, was Haapsalu, a sheltered bay on the western coast. There the bishop began building a castle and cathedral that would take three centuries to complete.

The western side of the castle holds a 29-meter watchtower from the 13th century, later used as a bell tower; the curtain walls were eventually raised to 15 meters. The cathedral inside the walls, built of local limestone, is one of the largest single-nave churches in the Nordic and Baltic region, a great cool space whose acoustics turn even an ordinary voice into something architectural. The inner trenches and blindages, built for cannon emplacements and as bomb shelters, date to the Livonian War of 1558-1582, when the stronghold was severely damaged. Walls of the small inner castle and the outer fortification were left partly destroyed. On 23 March 1688 a fire took the sheet-copper roof; restored quickly, it was destroyed again in a 1726 storm. The shrunken congregation could not pay for another rebuild and moved to the town church. In the 19th century the ruins began their next life as a romantic castle park.

The legend is medieval in setting and very old in spirit. According to the story, a young woman disguised herself as a chorister to be near the canon she loved. When she was discovered, the bishop summoned his council, and the council decided that she should be immured alive in the chapel wall. The canon was thrown into prison and starved to death. The builders left a small cavity. The girl was placed inside with a piece of bread and a mug of water, and the wall was sealed. Her cries were heard for some time. Her soul, the legend continues, has never found peace, and on August full moons she appears on the baptistery window. The story belongs to the broader European tradition of immurement legends. What makes Haapsalu's version distinctive is the stubborn local insistence that the apparition is real and visible, and that watching for it together is something a community does every August.

By the 19th century, Haapsalu had become a fashionable spa town. The Russian imperial family visited. Healing mud baths drew patients from across the empire. In the summer of 1867, the young Pyotr Ilyich Tchaikovsky spent time here with his brothers Modest and Anatoly, walking along the Promenade and the seafront. The town's wooden bench dedicated to him, the Tchaikovsky Bench, plays his "Chant sans paroles" from Souvenir de Hapsal at the press of a button. The composer wrote pieces inspired by his time here, including the piano work Souvenir de Hapsal. The town that built itself around a half-ruined episcopal castle and a ghost story turned out to be a place where Russian Romantic music could find a quiet voice.

Inside the castle today stands the Mother's Altar, dedicated to Estonian mothers murdered or deported during the Soviet occupations. It is a small, modern monument in an ancient space, and it is part of why Haapsalu's castle does not feel like a museum. The medieval walls hold the legend of the White Lady. The 19th century left the spa-town atmosphere and the music. The 20th century left the Mother's Altar. The 21st century brings the August Blues Festival and the White Lady Days. The castle does not so much preserve history as keep adding chapters to it.