Caerhays is what the Georgians called a mock castle - a country house dressed in defensive clothing, with crenellations and towers but no military function. The exterior was built of rough stone quarried from the immediate neighbourhood, giving the walls a colour that matches the headland. The front entrance is 160 feet long with a southern exposure, raised on an embattled terrace. Painted glass adorns the windows of the dining room, staircase and entrance hall. Other Grade I listed buildings associated with the castle include a folly tower attached to the west and east, the Higher Lodge, the Lower Lodge with attached screen walls, and the service buildings to the south-west. Nash designed Caerhays close to the site of the original manor house, which had itself been expanded during the reign of Henry VIII. The house that was demolished to make room for the castle was not modest. The castle that replaced it was simply grander.

The earliest record of the place-name is Karihaes in 1259, with Carihays appearing in 1379 - the original meaning of the name is lost. In the early Middle Ages the manor belonged to the Arundell family, who held estates across Cornwall. Around 1379 it passed by marriage to the Trevanion family when Robert Trevanion married Johanna Arundell, daughter and heiress of Rudolph Arundell of Caerhays. The Trevanions held it for over four hundred years. John Trevanion inherited in 1703 and improved the existing manor house, developing the early gardens. William Trevanion died in 1767, leaving the estate to his sister's son John Bettesworth, whose son commissioned Nash. After Bettesworth-Trevanion's flight to Paris, Michael Williams II - of a Cornish mining and banking family - bought the estate from his creditors in 1854. The Williamses have held it ever since. Charles Williams owned the castle as of 2023.

When John Charles Williams inherited Caerhays in 1880 he was eighteen years old. He became one of the great Edwardian plantsmen and the principal reason Caerhays is internationally famous today. Williams sponsored plant-hunting expeditions to China, then opening to Western botanists for the first time. Seeds collected by Ernest Henry Wilson in 1903 - Wilson was an English plant collector working for the Veitch nursery and then the Arnold Arboretum - were donated to J.C. Williams by Harry Veitch and planted at Caerhays. The seeds included magnolias, rhododendrons, camellias and other Chinese woody plants that had never been cultivated in Europe before. The Williamses developed new hybrids - the famous Williamsii camellias, named for the family, are still sold worldwide. By 1917 the garden held over 250 types of rhododendron. The magnolia collection is now accredited under the National Plant Collection scheme operated by Plant Heritage.

The castle garden covers almost 120 acres, traversed by four named routes and connected by trails, grassy paths and steps. Six hundred varieties of plants grow there, including some of the oldest cultivated magnolias in Britain. Late February to early May is the spectacular season - the camellias bloom first, then the early magnolias, then the rhododendrons, before the garden settles into the deep green of Cornish summer. Walking the trails on a misty April morning you understand why nineteenth-century plant hunters risked malaria and bandits in remote Chinese valleys - the magnolias rise four storeys tall with flowers the size of dinner plates, white and pink and palest yellow against the grey Cornish sky. The garden is open to the public for several weeks each spring.

Caerhays has done service as a film set more than once. The 1979 BBC miniseries adaptation of Rebecca used the castle for its Manderley exterior shots - it stands in for Daphne du Maurier's haunted, doomed Cornish house with very little adjustment. Tim Burton's 2016 film Miss Peregrine's Home for Peculiar Children filmed for three months at the estate and on Porthluney Cove below. Caerhays also features in the later Poldark novels, where the son of Ross and Demelza Poldark meets a love interest at the castle - and Winston Graham's writing stresses both the magnificence of the place and the ruinous cost the Trevanions paid to build it. The castle is private; the gardens open to the public when the magnolias bloom. The cove beneath is open beach. The house Nash designed for a man who could not afford it has, in its way, paid for itself many times over.