Walk along the Thames at Blackfriars today and you would not know that another river joined the Thames here. The Fleet, once one of London's larger rivers, has been reduced to a culvert that emerges quietly under Blackfriars Bridge. In Roman times the Fleet was the western boundary of the city, and the junction of two rivers made the corner strategically irresistible. The north wall of the first Baynard's Castle used as its foundation the Roman river wall from the third century, a course of tile-and-Roman-brick masonry visible to archaeologists. The chronicler Richard of Cirencester suggests that King Cnut spent Christmas at a fort on this site in 1017, where he had Eadric Streona, the Ealdorman of Mercia, executed - reputedly after an argument over a game of chess. Long before Ralph Baynard arrived, this spot had a habit of seeing power exercised and lives ended.

Ralph Baynard arrived with William the Conqueror and was made the first feudal baron of Little Dunmow in Essex. The castle he built passed to his son Geoffrey and grandson William Baynard, who lost the lands in the early twelfth century for backing Robert Curthose against Henry I. After a brief spell in royal hands, the castle ended up with the king's steward Robert Fitz Richard, and eventually with his grandson Robert Fitzwalter. Fitzwalter became the leader of the baronial revolt against King John that produced the Magna Carta in 1215. The Chronicle of Dunmow tells a more colourful story: that King John desired Fitzwalter's daughter Matilda the Fair, also known as Maid Marian Fitzwalter, the real-life basis for the Maid Marian of Robin Hood legend, and that Fitzwalter took up arms in her defence. In 1213, before all of that, John ordered the castle demolished. The site was sold in 1276 to make way for Blackfriars Monastery.

Even after the Norman keep was gone, the lord of Castle Baynard held a peculiar place among the nobility of London. The tenure carried the right - claimed in long legal battles - to be the Chief Banneret of London, the man who carried the banner of the city into battle and led its forces. In 1136 Robert Fitz Richard had claimed lordship of the Thames from London to Staines as the king's banner-bearer and guardian of the city. In times of peace, the soke of Castle Baynard - roughly the modern Castle Baynard Ward - held a court that sentenced criminals convicted before the Lord Mayor at the Guildhall, kept a prison and stocks, and dealt with traitors in its own grim way: they were tied to a post at Wood Wharf and left for the rising tide to drown.

About a hundred years after King John's demolition, a new mansion was built on land reclaimed from the Thames just south-east of the first castle. Humphrey, Duke of Gloucester, rebuilt the house after a fire in 1428 and used it to house his family before the Battle of Barnet during the Wars of the Roses. By the middle of the fifteenth century the house had become the London headquarters of the House of York. On 3 March 1461, in the great hall of Baynard's Castle, Edward IV was proclaimed king. In 1501, Henry VII essentially rebuilt the place as a royal palace - according to the chronicler John Stow, not so strongly fortified as before, but far more beautiful and commodious for the entertainment of any prince or great estate. Henry VIII gave it to Katherine of Aragon on the eve of their wedding. Margaret Tudor, widow of James IV of Scotland, stayed here in May 1516. Katherine Howard journeyed here in May 1541, before her own marriage to Henry VIII ended at the Tower.

In 1551, during the reign of the young Edward VI, the castle was granted to William Herbert, Earl of Pembroke - brother-in-law to Henry's last queen, Katherine Parr. Pembroke built a great extension around a second courtyard, clearly visible on Wenceslaus Hollar's view of London before the Great Fire, with a large gateway on the south side, a two-arched bridge, and steps down to the river. It was at Baynard's Castle in 1553 that Pembroke proclaimed Mary I queen, helping to overturn the brief reign of Lady Jane Grey. The house remained in the Herbert family through the Civil War - they took the side of Parliament - and after the Restoration in 1660 it passed to a Royalist, Francis Talbot, the eleventh Earl of Shrewsbury.

The Great Fire of London in 1666 ended Baynard's Castle as it ended much of medieval London. The palace burned. Some fragments survived into the nineteenth century - one or possibly two towers stood until they were pulled down to make way for warehouses of the Carron Company; by 1878 the site was owned by the Castle Baynard Copper Company. Most of the archaeological evidence we have for the second castle comes from excavations between 1972 and 1975, before the construction of Baynard House, the BT office block that now sits where the Tudor palace once stood. Two east-west limestone walls were found, parts of the north wing, a north gate and gate-tower, the cobbled entrance off Thames Street. The London Archaeological Archive holds the records under codes BC72, UT74, BC74, BC75, and BYD81.

Today there is nothing visible above ground to tell you that two royal palaces and a Norman keep once occupied this corner of the City. The river itself has been pushed back, the Embankment built out into what used to be the Thames foreshore. Blackfriars station rises over the river to the west; St Paul's dominates the skyline a short walk to the north-east. Sir Walter Besant, writing in 1903, said there was no house in London more interesting than Baynard's. He was probably right. From the air today, the site is unremarkable - a strip of office buildings between the river and Queen Victoria Street. But every brick of Baynard House sits on the place where Edward IV was proclaimed king and Mary I was proclaimed queen, where Cnut may have spent a strange Christmas in 1017, and where Robert Fitzwalter set out to defend his daughter's honour and ended up with Magna Carta.