Archaeological excavations in the 1920s uncovered evidence of occupation stretching back to the late Bronze Age, roughly 900 to 500 BC. The Romans built a signal station on the headland, a square wooden structure on stone foundations measuring about 33 meters across, with a gatehouse and an outer ditch, part of a chain of coastal watchtowers monitoring the North Sea approaches. The signal station was probably destroyed in the upheavals of the fifth century. An Anglo-Scandinavian chapel was later built on the same site. One tradition claims that Harald Hardrada's forces burned a Viking settlement around the harbor in 1066 and lit a great bonfire on the headland to hurl burning brands at the villagers below, though no archaeological evidence supports this. What is clear is that the headland's natural defenses, steep cliffs on three seaward sides and a narrow isthmus connecting it to the mainland, made it an obvious place to build a fortress.

William le Gros, Count of Aumale, built the first wooden fortification around 1138 after his victory at the Battle of the Standard. When Henry II came to the throne, he demanded the return of all royal castles and demolished Aumale's wooden fort. From about 1157, Henry rebuilt in stone, spending 682 pounds over his reign, a significant sum when the average royal annual income was roughly 10,000 pounds. The three-story keep, with walls over fifteen feet thick at the base, rose within the inner bailey. But it was King John who lavished the most money on Scarborough. John spent 2,291 pounds on the castle, more than on any other fortress in England, building curtain walls, a new great hall called the King's Chambers, and visiting four times during his reign. The Pipe Rolls record that John invested in 95 castles; Scarborough received the most. Henry III added a barbican with flanking towers. Edward I held court here. Edward II made his controversial favorite, Piers Gaveston, governor of the castle in 1312, only for barons to besiege and capture both Gaveston and the fortress within weeks.

The castle changed hands seven times between 1642 and 1648 during the English Civil War. The first siege began in February 1645, when Sir John Meldrum's Parliamentary forces took the town and cut off the castle by land and sea. The bombardment partially destroyed the keep, but the outer walls held, and Meldrum himself was killed in hand-to-hand fighting around the barbican gateway. The 500-strong Royalist garrison held out until July, when scurvy, starvation, lack of water, and only 25 men fit to fight forced a surrender. Roughly half the original defenders emerged alive. The castle was repaired and rearmed for Parliament, but in 1648 the new governor, Matthew Boynton, declared for the king when his soldiers went unpaid, triggering a second siege. Winter proved as effective as artillery. By December, Parliamentary control was restored. The shell of the keep survives today, minus its west wall, blown away by cannon fire. George Fox, founder of the Quakers, was later imprisoned here. The castle served as a prison, a powder magazine, and a garrison post through the Jacobite rising of 1745 and the Napoleonic Wars.

In the second half of the nineteenth century, the castle began its final transformation: from fortress to tourist attraction. The Ministry of Works took ownership in 1920. Foundations of a medieval hall, excavated in 1888, were opened to visitors. The eighteenth-century Master Gunner's House became a museum, its centerpiece the Bronze Age sword found on the headland in 1980, a reminder that this site was considered worth defending long before anyone called it a castle. English Heritage now manages the site. Visitors climb the barbican, walk the curtain wall, and stand in the roofless keep looking out at the same North Sea that Roman sentries watched from their signal tower. The 150-foot well in the inner bailey still descends into the rock. The red brickwork of the eighteenth-century barracks, badly damaged by German naval shelling in 1914, is visible against the older stone from Scarborough's South Bay. Three millennia of human activity layered onto a single promontory, each era leaving its mark, none quite erasing what came before.