The story of Sherborne begins centuries before Raleigh. The building now known as Sherborne Old Castle was constructed in the twelfth century as the fortified palace of Roger de Caen, Bishop of Salisbury and Chancellor of England. During the Anarchy of the 1140s, when Stephen and Matilda fought for the English throne, the castle was captured by Robert, Earl of Gloucester, who considered it "the master-key of the whole kingdom." Its position in the rolling Dorset landscape gave it strategic importance far beyond its size. For four hundred years, it passed through ecclesiastical and royal hands before Raleigh arrived and decided its future.

Raleigh completed his lodge in 1594. It was an unusual building for its time, with four polygonal corner turrets that gestured toward military defence without actually providing any, a nod to the medieval castle visible across the park. The entrance was cleverly disguised in one corner tower to preserve the facade's symmetry. But Raleigh's enjoyment was brief. His imprisonment in the Tower of London left the estate vulnerable, and King James first leased it to Robert Carr, then sold it outright to Sir John Digby, 1st Earl of Bristol, in 1617. The Digby family added four wings in the 1620s, transforming Raleigh's compact lodge into the mansion now known as Sherborne Castle.

The English Civil War tore Sherborne apart, literally. Lord Digby was a Royalist advisor to Charles I, and the area was strongly Royalist. The fortified old castle changed hands twice: captured by Parliamentarians in September 1642, recaptured by Royalists in February 1643. In August 1645, Sir Thomas Fairfax and the New Model Army laid siege, subjecting the castle to heavy bombardment and mining until the Royalist garrison under Sir Lewis Dyve surrendered on 17 August. Two months later, Parliament ordered the old castle slighted. Its walls were torn down to prevent any future military use, leaving the picturesque ruin that stands today. Raleigh's lodge, being neither fortified nor strategically significant, survived.

In the eighteenth century, the Digby heirs turned destruction into design. The 5th Lord Digby laid out grounds that Alexander Pope praised, and in 1753 Capability Brown created the lake that now separates the ruined old castle from the lived-in new one. The ruins became a deliberate feature of the garden, picturesque among the trees across the water. George III visited in 1789. During the First World War, the mansion served as a Red Cross hospital; during the Second, it housed the commando headquarters for the D-Day landings. The Wingfield Digby family, who inherited in 1856, still own the property. Both the mansion and the ruins hold Grade I listed status, and the gardens are open to the public for much of the year, a landscape where the wreckage of civil war has become part of the beauty.