Westhove first appears in writing in 1277, in a charter from Count Floris V of Holland confirming the castle as the property of the powerful Premonstratensian abbey at Middelburg. How old the building was even then is not certain — the charter recognises something that already existed. In 1290, Floris V and Wolfert I van Borselen, lord of Veere, signed the Traktaat van Westhove at the castle, fixing rights and obligations between count and noble. The abbot of Middelburg was no minor cleric: he chaired the States of Zeeland and of Walcheren, and the assembly met in his abbey. Westhove was the abbot's country retreat, the place he went to escape the politics he otherwise controlled. Every time a new abbot arrived, the castle was symbolically renamed, an ecclesiastical custom that turned the building into a kind of changing nameplate for whoever ruled the spiritual life of Zeeland.

The list of people received at Westhove reads like a sampling of the Burgundian-Habsburg ascendancy. Charles the Bold came. So did the Count and Countess of Charolais in 1460. Philip the Fair of Austria stayed with his wife in 1500. Their five-year-old son visited in 1505, came back in 1540 — by which time he had become the Holy Roman Emperor Charles V, the most powerful man in Christendom. His sister Mary of Hungary stopped by in 1547. Two and a half centuries later Prince William V dropped in on 1 July 1786. Then, on 22 August 1894, the Queen Dowager Regent Emma arrived with her thirteen-year-old daughter, Wilhelmina, who would in a few years be queen and would eventually reign for fifty-eight years. The visitor's book at Westhove, written across seven centuries, is essentially a fragment of European royal genealogy.

In 1559 Middelburg became a bishop's see, and from then until 1572 a bishop ran Westhove. That run ended badly. The Eighty Years' War was on. The Siege of Middelburg pitted Spanish defenders against the watergeuzen, the Sea Beggars — Dutch Calvinist privateers who fought their war on tide and dyke and won it by knowing the coast better than their enemies did. Bartholt Entens van Mentheda, a Groningen nobleman and captain of the Sea Beggars, had already taken Dordrecht and brought his men south to Zeeland. The soldiers garrisoning Westhove surrendered to him. On 27 August 1572 he set the castle on fire. Walcheren passed into the hands of William of Orange, the founder of the Dutch Republic. The abbey's holdings were sold off, and Westhove began a long second life as private property — first under Colonel Heijnrik Balfour, then under the diplomat Pierre de Loiseleur de Villiers, who served as counsel to William of Orange himself. De Villiers died in 1590 and was buried at the St Pieterskerk in Middelburg.

Through the seventeenth and eighteenth centuries, Westhove passed from one Zeeland family to another, gathering the small details that turn a fortified house into a country estate. The Boreels — Jacob and his son Willem (1591–1668), ambassador to France — held it for a time. Then it went to the Van Reigersbergs, who left a heron-shaped weather vane on the roof, dating from around 1725, that still spins in the dune wind. Jacoba van den Brandde inherited the place, married Johan Adriaan van de Perre, died childless, but left behind an orangery — the elegant glass-walled greenhouse that Dutch grandees built to overwinter their lemon trees and orange trees. That orangery still stands; today it houses the Terra Maris museum, which tells the natural history of Zeeland. In 1799 the estate passed to Wilhelmina Carolina van den Brande, who married Adriaan Slicher; they built the large reflecting pool on the north side of the castle. Their son sold the property in turn.

In 1880 Judge Jacques Phoenix Boddaert of Middelburg, who had bought the estate and lavished maintenance on it, sold Westhove to his sister, who in 1899 reopened it as a convalescent home — a Dutch sanatorium for patients sent to recover in the sea air. The dunes of Walcheren and the pine woods that ring the property were considered medicinal in an age that took weather seriously as medicine. The convalescent function continued in various forms through the twentieth century, surviving even the 1944 inundation of Walcheren, which flooded much of the island but spared the high dune ground around the castle. Eventually the foundation that ran it closed, and Westhove found its present life: a StayOkay hostel in the castle itself and in the surrounding coach houses, the Terra Maris museum in the orangery, and a small reproduction of a medieval motte-and-bailey castle in the garden for visitors who want to see what the original might have looked like before stone replaced timber. The attic is still off-limits. Everything else is open to anyone with a reservation and a backpack.