The first fortress on this rock above the Pelcznica river was destroyed by Bohemian forces in 1263. A generation later, the Silesian duke Bolko I the Strict began a new castle here between 1288 and 1292, taking up residence and adding 'Lord of Ksiaz' to his titles. The site is dramatic in a way that medieval lords prized: a cliff edge above a wooded gorge, with views down the valley and good defensive geometry. After the Świdnica Piast line died out in 1368, the castle passed through Bohemian and then Habsburg hands, and in 1466 Hans von Schellendorf received it from the Bohemian crown and renamed it Schloss Furstenstein. For most of its German history it would carry that name - the German aristocracy treating it as a piece of inherited furniture rather than a castle of state.

From 1509 the castle belonged to the Hochberg family, who held it for more than four centuries. They expanded it relentlessly: the Baroque additions in the early 18th century, the neo-Gothic tower in the 19th, the great east wing in the early 20th. By 1909 Ksiaz contained over 400 rooms across roughly 11,000 square meters, surrounded by terraced gardens scaled to a hereditary self-importance the Hochbergs never seem to have doubted. The most famous resident was Mary Theresa Olivia Cornwallis-West - Princess Daisy of Pless - the British-born wife of Hans Heinrich XV. Daisy was a memoirist, a confidante of Edward VII, and during World War I a Red Cross nurse who criticized German conduct in print. She died in Wałbrzych in 1943, mostly impoverished, having watched her family's empire dissolve and her castle be requisitioned by the very regime she had warned against.

In 1941 the Nazi state effectively confiscated Ksiaz from the Hochbergs. By 1943 the SS had begun the Project Riese excavations beneath it. The forced laborers who did the work were prisoners of the Furstenstein subcamp of Gross-Rosen - among them Italian, Polish, Hungarian, and Jewish men deported from the death camps when their labor was deemed more valuable than their immediate murder. Mortality was severe; the documentation that survives describes bodies removed quietly to mass graves. Above ground, Albert Speer's architects modified the castle interiors for what was reportedly intended as a Fuhrer headquarters; entire floors were stripped and rebuilt. The work was not finished when the Red Army arrived in 1945. The Soviet occupation that followed completed what the Nazis had begun in destruction, looting the interiors and burning much of what could not be carried off. A communist-Polish memorial today marks the site of the subcamp where the laborers died.

The Polish state took over Ksiaz after the war and slowly began restoring it. The interiors had to be reconstructed almost from photographs - the Maximilian Hall, the Baroque Salon, the Chinese Room - using craftspeople whose grandfathers had worked the original surfaces. A 2014 attic fire destroyed nearly 500 square meters of roof and slowed the work again. But the castle today draws over 300,000 visitors a year, and in 2021 the National Bank of Poland minted a special 5-zloty coin featuring it as part of a 'Discover Poland' series. The Pelcznica gorge below is preserved as Ksiaz Landscape Park, the Sudeten foothills behind it visible from any high window. Walk the grounds early enough and the view is what Bolko I the Strict bought it for - a defensible rock over a winding river - the rest, all 400 rooms and seven and a half centuries of additions, an accumulation.