Clovelly has been continuously owned by three families since the middle of the thirteenth century, which makes it one of the longest-held private estates in England. The Cary family bought the manor in the late 1300s under Richard II and held it for four hundred years. The Hamlyn family bought it in 1738. The Rous family inherited it from the Hamlyns by marriage and still owns it. The current proprietor is John Rous, who inherited from his mother in 1983 and lives at Clovelly Court above the village. He is the only son of Keith Rous, the fifth Earl of Stradbroke, by his second marriage to Mary Asquith — who was the granddaughter of the former Prime Minister Herbert Henry Asquith. Clovelly is therefore owned by a man who is technically the grandson, by marriage, of a Prime Minister of the United Kingdom. He runs the place via the Clovelly Estate Company. Every cottage in the village belongs to it. Every cottage is rented from it.

Clovelly was an agricultural parish for most of its history. Then in the late sixteenth century the squire George Cary spent two thousand pounds — a small fortune at the time — building a stone breakwater out into Clovelly Bay and turning the cove into the only safe harbour between Appledore and Boscastle, a hundred miles of otherwise hostile coast. Once the breakwater was there, the village had a reason to exist beyond farming. Cary built fish cellars and warehouses at the foot of the cliff. He built cottages along the path of the only stream that connected the cliff-top to the bay. The cobbled street that runs through Clovelly today follows that stream-cut. The village's shape is the shape of the water that ran down to the sea before anyone built anything here.

Almost everything you see when you walk through Clovelly today is owed to one woman. Christine Hamlyn, who inherited the estate in 1884 and ran it until her death in 1936, spent decades restoring and rebuilding the cottages along the main street — keeping them in seventeenth-century vernacular style, banning anything that would have looked out of place. She built a new car park up on the plateau above the village expressly to keep cars out of the streets. By the 1930s, fifty thousand cars a year were driving up to that car park to have their occupants walk down to look at her village. In the 2020s, the figure is one hundred and fifty thousand visitors a year, paying nine pounds fifty for entry. The proceeds go back into the maintenance of the houses. The village is essentially a self-funding museum that happens to also be lived in.

The novelist Charles Kingsley lived in Clovelly as a child from 1831 to 1836 — his father was the rector. In 1855, by then a famous author, Kingsley published Westward Ho!, a swashbuckling Elizabethan novel set partly in Clovelly and partly in the broader West Country. The book sold enormously. It was the Victorian equivalent of a viral promotional film for the village. Within a generation, Clovelly was on every Devon tourist itinerary. The Turner painting of Clovelly Harbour from around 1822 — now in the National Gallery of Ireland — had already given the village its romantic visual identity. Dickens described Clovelly in A Message from the Sea in 1860. Rex Whistler painted miniatures of the village onto a Wedgwood china service. The actor Joss Ackland lived in Higher Clovelly from 1989 until his death in 2023 and was, for the last decades of his life, a kind of unofficial ambassador for the place.

On the 28th of October 1838 — a Sunday — twelve Clovelly fishing vessels left the harbour with twenty-six men aboard. A ferocious storm caught them at sea. One vessel returned. The other eleven, and the men who had been aboard them, did not. The village's official population at the next census was just over four hundred. The loss of more than twenty working-age men in a single night reshaped the community for a generation. The Shipwrecked Mariners' Society was founded the following year specifically to support the widows and orphans left by such losses, and it remains active today. There is no large memorial in the village to the men who died that night. There is the Church of All Saints up on the hill — Grade I listed, parts still Norman — and the gravestones of fishermen of many other generations. The 1838 dead are mostly anonymous in the parish register, lost together at the moment when the parish had no time to record who they had been.