The Annesleys arrived in this corner of County Down in the early eighteenth century and started planting in 1740 - the year considered the foundation of the arboretum. Hugh Annesley, 5th Earl Annesley, who inherited in 1874, was the great expander. With his head gardener he assembled what is now described as "one of the most remarkable tree collections not only in Ireland but in the whole of Europe." The original 12.5-acre walled garden remained the heart of the arboretum, but Annesley extended planting outward until the formal grounds covered 120 acres. By the end of the nineteenth century there were over 1,800 species of trees and shrubs growing here. More than 700 taxa still survive, including at least thirty champion trees - specimens that are the largest of their kind recorded in Britain and Ireland.

In 1853 the English plant collector William Lobb returned from California with a haul of seeds collected from the great Sierra Nevada giant sequoias - Sequoiadendron giganteum - the largest trees on Earth. Lobb sold his seeds to the Veitch nursery in Exeter; saplings were grown on for three years. In 1856 the Annesleys planted a group of these saplings at Castlewellan, among the earliest cultivated sequoias anywhere outside California. They are still standing. One of them has done something genuinely peculiar: it has developed nineteen separate trunks, a multi-stemmed form almost never seen in cultivated giant sequoias. That tree was voted Northern Ireland's Tree of the Year in 2018. Tree-creepers - small woodland birds that climb tree trunks looking for insects - now nest in burrows dug into the sequoias' famously soft, fibrous bark.

Standing close to the entrance of the arboretum, overlooking Castlewellan Lake, is Castlewellan Castle - a Scottish baronial pile of locally quarried granite built between 1856 and 1858. The architect was the Scotsman William Burn, then near the end of a long career as the leading designer of Scots baronial country houses; the patron was the 4th Earl Annesley, who never married and died in 1874. The estate passed to his brother Hugh, then to Hugh's son Francis, the 6th Earl. Francis was killed in the First World War in 1914 at the age of 30. The earldom went to a cousin; the Castlewellan estate went to Francis's sister Lady Mabel Annesley, an artist of some accomplishment, who eventually passed it to her son Gerald. In 1967, Gerald sold the estate to the Forestry Service of the Department of Agriculture. The park opened to the public the same year.

Between 2000 and 2001, in the years after the Good Friday Agreement, 6,000 yew trees were planted at Castlewellan to form a maze. The yews were planted by volunteers drawn from across Northern Ireland - Catholic and Protestant, unionist and nationalist - in a symbolic act of building something together. When it was completed in 2001 the Peace Maze was, briefly, one of the largest permanent hedge mazes in the world. The yews are slow-growing. The maze has taken its time to mature. The point was never speed. The maze sits inside a 460-hectare park in a region that needed something it could plant together and watch grow.

At the centre of the arboretum, the original twelve-and-a-half acre walled garden of 1740 still functions as it was designed to function - protected from the worst of the County Down weather by high stone walls, drained by the same gravity-fed system, planted with formal paths and ornamental fountains. The notable specimens read like a gardener's wish-list: the original Castlewellan Gold Leylandii, a Ginkgo biloba, the handkerchief tree (Davidia involucrata, named for the fluttering white bracts that hang like handkerchiefs in spring), magnolias, rhododendrons, conifers. By the late 1990s the gardens had begun to decline; resources were short and the Moorish Tower at the head of the garden had fallen into ruin. Between 2012 and 2014 partial restoration was done. In 2021 a £5.5 million programme of further work was announced, funded jointly by the National Lottery Heritage Fund and Newry, Mourne and Down District Council.

The park covers 460 hectares, with a 40-hectare lake reflecting the castle and the encircling forest. The Mourne Mountains rise to the south, granite peaks that gave the castle its building stone. Beyond the lake, paths wind through plantings of trees from Asia, Australasia, North America and South America - Athrotaxis from Tasmania, Japanese maples turning crimson in October, the giant sequoias growing slowly toward whatever age they will reach in a Northern Irish climate that is much gentler, though much wetter, than their native Sierra Nevada. Treecreepers work the sequoia bark. Yew leaves darken the Peace Maze. The Castlewellan Gold Leylandii, the great mutant accident of the place, continues its slow conquest of suburban hedges across the British Isles. A garden begun in 1740 keeps growing.