The exact date of construction is impossible to recover. Most scholars now ascribe it to the Florentine Acciaioli family, who held the Duchy of Athens from 1388 until the Ottoman conquest in 1458 and converted the area near the Propylaia into a palace. Some, like the medievalist Peter Lock, argue it might equally be the work of the earlier de la Roche dynasty, the thirteenth-century French dukes whose residence on the same site has left no architectural trace of its own. Whoever built it, the tower served whichever regime was in charge. It was a watchtower with a view across the central plain of Attica to Acrocorinth in the Peloponnese - a 1908 history by William Miller suggests a beacon on its north turret could be seen sixty kilometers away. It was a beacon. Under the Ottomans it was a salt store and a prison. The Turkish word for tower, kule, gave it the local name Goulas. Chaucer may have heard about it. Some scholars believe the great tower in the Knight's Tale, where Palamon is imprisoned in the palace of the Duke of Athens, was modeled on this one.

When the Greek War of Independence broke out in 1821, the tower took on a darker role. The Ottoman authorities used it to imprison twelve Athenian notables as hostages, of whom nine were executed during the 1821-22 siege of the Acropolis by the Greek rebels. Three escaped. After the rebels retook the rock, they raised the tower's height to extend its sightlines for an observation post. In 1825 it received its most famous prisoner. Odysseas Androutsos was a captain of the revolution who had defected to the Ottoman side under circumstances historians still debate, was captured by his former Greek comrades, and was held in this tower in chains. He was tortured. He died there. His body was found at the foot of the tower, with a rope hanging from a window above - which observers continued to point out to visitors as evidence of a failed escape attempt for the next fifteen years. After 1825 some Greeks called the building "Odysseus's Tower."

After independence, the new Greek state had to decide what to do with the layers of history it had inherited on the Acropolis. King Otto's German architectural advisor Leo von Klenze, arriving in 1834, was generally in favor of removing post-classical structures from the site, but he made an exception for the medieval buildings near the Propylaia, finding them "picturesque." Other voices disagreed. The archaeologist Kyriakos Pittakis was an early advocate of demolition. Lysandros Kaftanzoglou, an architect and academic, called the tower "barbarian" in the Greek press and compared it to bird droppings. Foreign visitors labeled it a "barbarous sentinel" that interrupted the view of the Parthenon. The pressure was not universal: Filippos Ioannou, president of the Archaeological Society of Athens, opposed the demolition on the grounds that destroying medieval Greek heritage would only confirm foreign critics who said Greece neglected its medieval monuments. Théophile Gautier called the tower "an integral part of the Athenian horizon." In France, scholars like Jean Alexandre Buchon defended it as a symbol of continuity between ancient Greek and modern French civilization, since the Frankish lords had been French. The argument cut several ways.

Heinrich Schliemann, in 1874, was a wealthy German amateur who had made a fortune in business and now wanted to dig at the famous sites of the Greek world. He had been refused permits at Olympia and Mycenae. He proposed to fund the demolition of the Frankish Tower as what he called "a service to science" - and, more practically, as a gesture of goodwill that might help his case for an excavation permit. He offered the General Ephorate of Antiquities 12,000 francs. He paid an initial 4,000 drachmas to the sculptor Napoleone Martinelli, who would direct the work, and another 9,000 to the Archaeological Society. The committee voted in favor. King George I temporarily halted the demolition by royal order, prompting an indignant letter from Schliemann. The Ministry of Religious Affairs delayed it further. In the end the work went forward, finished in 1875, and cost Schliemann £465 - equivalent to roughly £47,000 in 2019 money. The historian Fani Mallouchou-Tufano has argued that the Great Eastern Crisis of 1875, with nationalist rebellions stirring in the Ottoman Balkans, encouraged Greeks to read the demolition as a reinforcement of national confidence. The British historian Edward Augustus Freeman called it "wanton destruction." William Miller called it "an act of vandalism unworthy of any people imbued with a sense of the continuity of history." The thousands of owls that had been roosting in the tower lost their home. No valuable inscriptions, the original archaeological pretext, were found in the rubble. The Frankish Tower remains the last building ever removed from the Acropolis.