The Vrijthof in Maastricht is dominated by two churches - the Romanesque basilica of Saint Servatius and the Gothic Sint-Janskerk next to it - and the square has been the heart of civic life for a millennium. The Spaans Gouvernement, on its western side, started life as ecclesiastical property. It was probably built for one of the canons of Saint Servatius and rebuilt in the early 16th century. The ground floor was largely blind then, broken only by an arched gateway into the courtyard. The three first-floor windows date from this rebuild - and the choice to put the Habsburg double-headed eagle and the arms of Castile in the central window made clear who, exactly, the canons were aligning themselves with. Charles V and Philip II stayed here on several occasions during their travels through the Netherlands. The name 'Spanish Government' stuck.

Shortly after the windows went in, a Renaissance arcade was added on the courtyard side, in a Liege style that echoes the main courtyard of the Prince-Bishops' Palace in Liege itself. The colonnade frieze carries the three medallions of Charles, Isabella, and Philip. It was probably in this building, during one of the Spanish governors' visits, that Alexander Farnese, Duke of Parma, signed the declaration outlawing William the Silent in 1580 - the formal break that made the leader of the Dutch Revolt a hunted man. A few years later, in 1584, William would be assassinated in Delft. The medallions in this courtyard watched the Habsburg century in the Low Countries through its sharpest turn.

By 1766 the Spanish were long gone and the building had a different kind of resident: Jean-Edme Dufour, a Parisian printer and editor who bought the house and turned it into a print shop. Maastricht, ruled jointly by the Dutch Republic and the prince-bishops of Liege, sat conveniently outside French jurisdiction. From this courtyard Dufour produced books that French authorities had banned, then smuggled them across the border into the country they were aimed at. For two decades, an Enlightenment book trade ran out of the same rooms where Habsburg governors had once held court. In the 18th and 19th centuries the interior was repeatedly altered; the once-blind ground-floor facade gained windows. In 1913 the building was sold off and partly demolished - one corner gave way to a 1923 bank building on St Jacobstraat. The architect-civil servant Victor de Stuers restored what remained and presented it to the city as a future museum.

The museum that the building had been waiting for finally arrived through a private gift. In 1954 the wealthy Hague couple Frederik Wagner and Ambrosina de Wit left their art collection to a foundation in Maastricht. From 1973 the bequest went on display as the Museum Spaans Gouvernement. A purpose-built pavilion in the courtyard housed boiseries rescued from a demolished 18th-century Maastricht mansion. The collection itself was a magpie's: Dutch and Flemish 17th-century paintings (Dirck van Baburen, Adriaen van de Venne, Nicolaes Berchem), Hague School landscapes by the brothers Maris, medieval and Renaissance sculpture, tapestries, antique coins, even artefacts from the Far East. Some of it has reportedly been sold off quietly at auctions since the 2010s to fund renovations, but the museum is private and not obliged to say. By 2014 the emphasis had shifted to local late-19th and early-20th-century artists. By 2019 the directors made a clean break: from now on, only photography.

A 2010-2012 renovation made the building two and a half times larger. The blind ground-floor facade was partly restored to its original austerity, the courtyard was roofed over, and a neighbouring building was annexed for exhibition space. When the museum reopened in 2012 it dedicated rooms to figures from Maastricht's history: Charles V, the smuggler-printer Dufour, the architect Mathias Soiron, the industrialist Petrus Regout, the draughtsman Philippe van Gulpen. Then, in 2019, came the renaming and the wholesale pivot to photography. The Fotomuseum aan het Vrijthof has no photographic collection of its own; every show is borrowed. It is also a private museum, supported partly by TEFAF - the Maastricht-based art fair that became a major sponsor in 2011 (one period room is now the TEFAF-zaal). The building keeps doing what it has always done: holding whatever the era thinks worth showing, in a courtyard where Charles V's pillars still point further beyond.