Medieval Wakefield had four chantry chapels, three dating from the 13th century, built on the roads leading out of town: Leeds to the north, Dewsbury to the west, York to the east. The Chantry of St John the Baptist stood where Wakefield Grammar School stands today. The Chapel of St Mary Magdalene was on Westgate. St Swithun's was near Clarke Hall on the York road. The fourth, St Mary the Virgin, was built in the 14th century on the new stone bridge over the Calder. All four chantries were paid for by the prayers of the living for the souls of the dead, an entire economy of medieval piety that came to an abrupt end at the Reformation. The Abolition of Chantries Acts of 1545 and 1547 dissolved them. Three of Wakefield's four were demolished. The bridge chapel could not be.

On 29 December 1460, the Battle of Wakefield was fought about a mile south of this bridge, and Edmund, Earl of Rutland, the second son of Richard, Duke of York, was killed near it while trying to escape. He was seventeen years old. Edward Hall's Tudor account, which Shakespeare reproduced in Henry VI Part 3, has Rutland murdered on Wakefield Bridge itself by Lord Clifford. Modern historians regard the bridge-killing story as largely apocryphal, but the chapel has carried the association ever since. The bridge had been there for a century when Rutland was cut down. It would survive another five centuries while every other building of his short life was demolished, rebuilt, demolished again. The chapel watched the Wars of the Roses begin in earnest a mile to the south, and it watches the traffic of modern Wakefield from the same spot now.

After the Reformation, the chapel did whatever was needed of a leftover medieval building. Warehouses use stone walls. Libraries use shelter. The bridge was widened in 1758 and again in 1797, and the chapel adapted both times. J. M. W. Turner painted it in watercolour in 1793, capturing the small ornate building against the open valley. By the 1840s the Oxford Movement had revived interest in medieval ecclesiastical architecture, and in 1842 the chapel was transferred to the Church of England. The Yorkshire Architectural Society commissioned George Gilbert Scott to restore it. Scott reconstructed the chapel almost entirely above pavement level, including a new west front carved in Caen stone. Both decisions are now considered errors. The Caen stone crumbled in the industrial atmosphere of nineteenth-century Wakefield and had to be replaced in gritstone in 1939 by Sir Charles Nicholson. The original richly carved medieval facade was moved to Kettlethorpe Hall, where it became the front of a folly boathouse. In 2014 the panels were brought back to Wakefield, to Thornes Park.

The west front today, in its 1939 gritstone, has buttresses at either end and three narrow doorways. Above the doors are five elaborately carved panels. Originally they showed the Annunciation, the Nativity, the Resurrection, the Ascension, and the Coronation of the Virgin, the great visual narrative of medieval Marian devotion. When Scott rebuilt the facade he replaced the fifth panel with the Descent of the Holy Ghost, a Pentecost scene that fit Victorian theology better. Three square-headed windows above the panels carry what stonemasons call flamboyant tracery, an undulating curvilinear pattern of carved stone. Inside, the chapel is small. A newel staircase in the northeast corner climbs to the roof. Another descends to a tiny crypt in the chapel's base, which is also the bridge's structural foundation. Four of the seven windows carry stained glass.

The Friends of Wakefield Chantry Chapel formed in 1991 from members of the Wakefield Historical Society, the Wakefield Civic Society, and St Andrew's Church. They have spent more than three decades raising money to repair the roof, repoint the stonework, rewire the building and install heating. A £30,000 stone renewal project carved six new heads for the south side of the chapel. At the architect David Greenwood's suggestion, four were modelled on living people: the Bishop of Wakefield, Lady St Oswald of Nostell Priory, the Rt Hon Walter Harrison, and Canon Bryan Ellis. The fifth head is Ray Perraudin, one of the Friends' founders. The sixth is a workman from the contractor William Anelay Ltd, whose name the records do not record. The chapel that began with prayers for the dead is now maintained by the descendants of the people who once prayed there, with their own faces carved into its walls.