Ryotan-ji traces its origins to 733 AD, when the celebrated Nara-period monk Gyoki is said to have established a temple called Jizo-ji on this site. But the temple's identity became inseparable from the Ii clan after the discovery of the infant Tomoyasu three centuries later. The monks' decision to raise the child set in motion a family line that would shape Japanese politics for the next millennium. The Ii clan served as local lords in the Iinoya valley for generations, and the temple became their bodaiji, the hereditary funerary temple where each generation's ashes and memorials were kept. By 1093, when Tomoyasu's own funeral was held here, the bond between temple and clan was already permanent. The temple took its current name, Ryotan-ji, in 1507, but its role as the spiritual anchor of the Ii family never wavered.

The Sengoku period brought chaos to the Iinoya valley. In 1560, Ii Naomori was killed in battle, and his funeral was held at Ryotan-ji. His successor Ii Naochika was also killed, leaving the clan in crisis. Into this void stepped Ii Naotora, celebrated in Japanese popular culture as one of the rare female lords of the Warring States era. She governed the Ii domain during one of its most perilous periods, protecting the young heir Ii Naomasa and ensuring the clan's survival. Both Naochika and Naotora are buried at Ryotan-ji. The temple itself was not spared from the era's violence. In 1570, invading forces of the Takeda clan burned it to the ground. It was rebuilt during the Edo period, though by then the Ii clan had been transferred to Omi Province. The Tokugawa shogunate granted the temple a fief of properties to sustain it after the loss of its longtime patrons.

Among Ryotan-ji's treasures is a set of three volumes from a Southern Song dynasty encyclopedia, printed in China in 1188. These books once belonged to Oda Nobunaga, the warlord who came closest to unifying Japan before his assassination at Honno-ji in 1582. How the volumes ended up at this Zen temple in the Iinoya valley is a story in itself: one of Nobunaga's sons became a monk, and the books passed through religious hands until they found their way here. The Japanese government designated these three surviving volumes as Important Cultural Properties in 1978, recognizing them as rare artifacts of Song-dynasty scholarship that survived eight centuries of war, fire, and upheaval.

On the north side of the Main Hall lies a garden that has been designated a National Place of Scenic Beauty since 1954. Designed by Kobori Enshu, one of the greatest garden artists of the early Edo period, the composition is an arrangement of carefully placed rock formations and tranquil ponds. Kobori Enshu was a polymath who served as a senior vassal of the Tokugawa Shogunate and excelled in tea ceremony, painting, poetry, and garden design. His most celebrated works include gardens at Nijo Castle and the Sento Imperial Palace in Kyoto. The garden at Ryotan-ji is a quieter affair, intimate in scale but precise in its geometry, a meditation in stone that has endured for roughly four centuries. As the head temple of one of fourteen autonomous branches of the Rinzai Zen school, Ryotan-ji is not generally open to the public, lending the garden an air of seclusion that feels earned rather than imposed.

Walking among the graves of the Ii clan at Ryotan-ji is to encounter an unbroken genealogy written in moss-covered stone. Forty generations lie here, from the foundling Tomoyasu to lords who served the Tokugawa. The temple's main image, a hibutsu statue of Kokuzo Bosatsu, remains hidden from public view, a secret Buddha that only adds to the sense that this place guards its stories carefully. The surrounding hills of northern Hamamatsu are thick with bamboo and cedar, and the Iinoya valley remains as quietly rural as it was when monks first raised a child they found beside a well. For the Ii clan, Ryotan-ji was never merely a temple. It was the place where their story began, and where every chapter since has been recorded in stone.