The church took its name from the pharos — the lighthouse tower — that stood beside it in the southern part of the Great Palace of Constantinople. It was first mentioned in the chronicle of Theophanes the Confessor in 769, when the future Emperor Leo IV married Irene of Athens there. The church stood close to the Chrysotriklinos, the golden throne room at the heart of the palace's ceremonial life, and to the adjacent imperial apartments. This proximity was not accidental. Emperor Leo V was assassinated in it in 820, which suggests how deeply embedded it was in the daily rhythms of imperial power. Following the end of iconoclasm, Emperor Michael III rebuilt and redecorated it extensively — the rededication probably took place around 864.

The accumulation of relics began early and continued for over two centuries. By 940, the chapel already held the Holy Lance — the spear that pierced Christ's side at the crucifixion — and a fragment of the True Cross. In 944, the Holy Mandylion arrived: the cloth believed to bear a miraculously imprinted image of Christ's face, brought from Edessa after a diplomatic agreement with the city's Muslim rulers. The right arm of St. John the Baptist came the following year. In the 960s, the sandals of Christ and the Holy Tile — the keramion, said to bear an impression of Christ's face transferred from the Mandylion — were added. In 1032, the collection gained the letter Christ was said to have written to King Abgar V of Edessa. By the end of the 12th century, the chapel also held the Crown of Thorns, a Holy Nail, Christ's clothes, his purple mantle, his reed cane, and a piece of his tombstone.

The scholar Cyril Mango's phrase captures what the Virgin of the Pharos had become by the time of its greatest magnificence: not simply a church but a statement about the nature of Byzantine imperial authority. The emperors who accumulated these relics were doing something deliberate. To possess the instruments of Christ's Passion was to position Constantinople as the truest successor to Jerusalem, and the Byzantine emperor as the truest guardian of Christendom. The keeper of the relics, Nicholas Mesarites, wrote a detailed description of the collection at the end of the 12th century — a document that survived to tell us what was there. The traveler Anthony of Novgorod visited and confirmed it. The chapel was, in the most literal sense, a concentration of everything that mattered to medieval Christian civilization.

The Fourth Crusade of 1204 ended it. When the Crusaders sacked Constantinople, the relic collection of the Virgin of the Pharos was not destroyed but dispersed — sold, given away, or carried off as plunder across Western Europe. The relics pertaining to the Passion were mostly acquired by King Louis IX of France, who built the Sainte-Chapelle in Paris between 1242 and 1248 specifically to house them, modeling his palace chapel directly on the Virgin of the Pharos. The idea was imitated again at Karlštejn Castle, built by Holy Roman Emperor Charles IV in the 14th century, whose relic chapel was tied to his self-presentation as a new Constantine. The Virgin of the Pharos is gone — the Great Palace itself is gone, buried under the streets of Sultanahmet — but its influence ripples through some of the most celebrated buildings in Europe.