The castle was known in Ottoman times by a Turkish designation, and also as 'Castle of the Plane Tree' — Φρούριο του Πλατάνου — and 'Castle of the Roots' — Ριζόκαστρο. It acquired its present name from a church dedicated to Saint Andrew that the Venetians built inside its walls when they occupied it, likely in honor of Admiral Andrea Pisani, who captured the fortress on 22 October 1717 during the Seventh Ottoman-Venetian War. The original Ottoman structure, completed around 1702, was a square with a bastion at each corner; the curtain walls ran about 200 meters per side, the walls were roughly a meter tall and 1.7 meters thick, reinforced with wooden stakes. A moat surrounded it, the excavated earth piled between the moat and the stone wall to form a second line of defense. It was modest — functional, not formidable.

The Venetians under Pisani took the castle without much difficulty in 1717; the Ottoman defenders apparently put up limited resistance against a force that arrived by sea. Once inside, the Venetians widened the moat, strengthened the defenses, and converted the garrison mosque into the Church of Saint Andrew. During their occupation they also reduced the size of the fortress by roughly half. Then, in 1797, the French Revolutionary government took control of Preveza along with the other former Venetian territories in the Ionian region. French rule lasted barely a year. In October 1798, Ali Pasha of Janina — the powerful and feared Ottoman governor of the western Greek mainland — routed the French garrison at the nearby Battle of Nicopolis. The castle barely featured in that fight. It was small, in poor condition, and surrounded by houses higher than its own walls.

Between 1807 and 1808, Ali Pasha undertook a thorough refortification of Preveza. He built a city wall, founded several new fortresses including St. George's Castle, and gave the Castle of Saint Andrew the form it still holds today. The most significant addition was a second, smaller wall on the seaward side, enclosing the harbor quarter of the city in a fortified circuit roughly 60 meters wide and 250 meters long, with three gates. He also demolished the Church of Saint Andrew and built a small mosque in its place — reversing, building by building, the Venetian mark on the fortress. The church that gave the castle its permanent name was gone, but the name itself remained.

The outer wall that Ali Pasha built was gradually demolished during the 20th century. The Greek army used the castle interior as a base for several decades, and in the process all of the original interior structures disappeared. What remains is the shell of the main castle — walls, bastions, the outline of the fortified circuit — sitting in the modern city of Preveza, accessible and unremarkable to anyone who doesn't know what the view of the narrow channel from its walls once meant. Across that channel, on the Actium promontory, planes land at LGPZ. Below the channel's surface, the undersea tunnel carries traffic between the two shores. The cypress tree that once marked the site is long gone.