Arthur Wellesley, 1st Duke of Wellington, died in 1852, and the Commissioners for Lighting the West Division of Southwark organised a memorial to stand at the southern end of London Bridge. Funds came from public subscription and contributions from the railway companies whose new lines were beginning to converge on that stretch of the Thames. Arthur Ashpitel, an established Gothic Revival architect, drew up a design in the Perpendicular Gothic style: three storeys, square in plan, the lower two walled, the upper storey open and intended to hold a statue of the Duke. One face of the ground level had a door with an ogee canopy, the others had arched windows. The first floor carried arched windows on every face beneath circular apertures for four illuminated clock dials, all surmounted by crocketed gables. The foundation stone was laid on 17 June 1854, and the tower was finished within six months. The money ran out before the statue could be commissioned. The Duke never made it to his memorial.

Even unfinished, the structure had a working role. The clock had been built for the Great Exhibition of 1851 by Bennett of Blackheath, the celebrated London watchmaker, who promised to maintain it for the rest of his life. The lower room of the tower was leased to the new electric telegraph network as a small office. Both arrangements failed. London Bridge in the mid-Victorian period carried a torrent of horse-drawn traffic, and the vibration shook Bennett's clock out of true so badly that it could never be relied upon. Worse was coming. The Charing Cross to London Bridge railway opened in 1864, followed by the 1863 Waterloo East viaduct, and the new viaducts overshadowed the little Gothic tower so completely that its memorial dignity was lost. The Metropolitan Police, looking at the bottleneck the tower created at the bridge's southern abutment, condemned it as a traffic obstruction. In 1867 it was disassembled.

George Burt, the Swanage stonemason and contractor who had made a fortune supplying Purbeck and Portland limestone to London builders, was in the right place at the right time. Burt was already shipping building materials between Swanage and the capital. He bought the dismantled tower, loaded the stones into one of his returning ships, and took it home. He left the clock mechanism behind. In Swanage, Burt gifted the stones to his fellow contractor Thomas Docwra, who erected the tower in the grounds of his house at Peveril Point. Reassembling it cost Docwra as much as the original construction had cost in London. He recorded the achievement by having his initials and the year carved into the base of the structure, where they remain visible today.

The tower as rebuilt in Swanage still carried Ashpitel's original spire, the crocketed Gothic finial that had pointed at the London sky. In 1904 a later owner removed it, possibly for stability, possibly for taste, leaving the truncated structure that visitors see today. The clock faces stare blankly out of their circular openings; no mechanism has rotated their hands for over a century and a half. The lower room that once held the London telegraph office is empty. But the tower itself remains where Docwra placed it, on the seafront south of the pier, and on 26 June 1952 it was given Grade II listed status as a Victorian Gothic structure of special interest.

Few buildings have had quite such a strange biography. Conceived as a memorial that never received its statue, it spent thirteen years failing in central London. It survived only because a Dorset quarryman saw the cost of the stones rather than the futility of the project. It has spent the last 159 years on the Swanage seafront, where it is now a familiar local landmark photographed by thousands of visitors a year, most of whom assume it has always stood there. The salt air has weathered the limestone gently. The crocketed gables still catch the light. Wellington himself was never represented, neither at London Bridge nor on the Dorset coast, but the tower built in his name has outlasted London Bridge as he knew it, and most of the railway viaducts that overshadowed it, and has become quietly Dorset's own.