The temple was established during the reign of Emperor Kangxi by Hsue Khui, a vice-general of the Penghu Naval Garrison. The site commands views across to Xiyu (Pescadores) and Baisha Island, and historical records indicate military forces had been stationed in this location since before the Dutch arrived in Taiwan in 1622. The Kingdom of Tungning and then the Qing Empire both maintained garrisons here, making the temple's founding inseparable from the navy's presence. In 1683, when Admiral Shi Lang defeated the Tungning fleet, he credited both the sea goddess Mazu and Guanyin -- the bodhisattva of compassion -- with boosting his soldiers' morale. That dual devotion persists: the temple serves both Guanyin and Long-Wang, the Dragon King, blending Buddhist and Taoist traditions in the manner typical of Taiwanese folk religion.

On March 29, 1885, French naval forces bombarded Penghu during the Sino-French War. The Guanyin Temple stood within the military zone near the Jinguitou Fortress and took direct hits. In the chaos that followed, soldiers from Guangdong and Zhejiang who were supposed to be defending Magong instead looted religious artifacts from the damaged temples -- an incident recorded with careful understatement in the 1878 Gazetteer of Penghu. The temple was left devastated. It took until 1891 for restoration to begin, when Wu Hung-lo, the first appointed general of the Penghu Naval Command, personally donated 500 taels of silver toward the effort. The current architectural form dates to a 1925 redesign by father-and-son builders Hsieh Chiang and Hsieh Tsu-nan.

When Japan took control of Taiwan and Penghu, the temple was absorbed into the Rinzai school of Zen Buddhism and renamed Hokosan Kannon-ji -- a Japanese Buddhist title that replaced its Chinese identity. By 1897, it had been incorporated as a branch temple under Myoshin-ji, the great head temple in Kyoto. The Japanese colonial government's support gave the seaside temple prominence, attracting worshippers from Japan and beyond. Intellectuals and Qing-era scholars gathered within its walls, and Phoenix Hall spirit-writing practices flourished. After World War II, however, the temple's fortunes reversed sharply. The Kuomintang government viewed temple-building as superstitious, and military units fleeing the Chinese Civil War were quartered in temples across Penghu for lack of housing. Around 1950, Master Kuang-tzu from Nanjing arrived and took over management.

The postwar years were lean. Financial hardship and government hostility to religious reconstruction left the temple crumbling. It was not until 1957 that a group of local citizens -- Kuo Shih-tou, Pao Wu, Tsai Chen, Hsu Teng-chueh, and Hsieh Keng-yin -- organized a reconstruction committee and mobilized donations from 38 villages across Penghu County. The renovation was completed on May 19, 1959, marking the temple's return to community life. Today the Guanyin Temple's profile has expanded well beyond devotion. Since 2003, its beachside location has made it the backdrop for Penghu's annual Firework Festival, held from April to June, which draws thousands of visitors to watch pyrotechnic displays reflected in the waters of the Taiwan Strait. The nearby Xiying Rainbow Bridge, lit in colors at night, frames the temple in a postcard scene that would astonish the Qing navy general who chose this spot for its military sightlines three centuries ago.