William Franklin Whittier was born in Vienna, Maine, and traveled to San Francisco via the Isthmus of Panama in 1854. He built his fortune in paint and white lead manufacturing, eventually becoming the major West Coast producer through a series of buyouts and the convenient deaths of two business partners -- one by accidental drowning in 1862, the other by natural causes in 1890. In 1893, he sold the firm to the Fuller family for $400,000 and used the 32 monthly installments to fund construction of his grand mansion. Designed by architect Edward Robinson Swain, the house cost $152,000 to build and furnish. Its steel-reinforced brick walls and Arizona red sandstone facade would prove their worth in the catastrophe that was coming -- the 1906 earthquake. The mansion was one of the few buildings in the city to survive essentially undamaged.

After Whittier's death in 1917, the mansion passed to his daughter Mattie Weir, who rented it to various tenants. One of them was Fritz Wiedemann, the German Consul General, who may have moved in as early as 1939. When the German Reich formally acquired the property in 1941, the mansion became the clearinghouse for German code documents and diplomatic mail flowing between Berlin and Nazi agents across North and South America. Wiedemann coordinated the activities of the German-American Bund from within, including overseeing secret ammunition stockpiles in New Jersey intended for use against the U.S. government if the order came. Both Wiedemann and Loeper were accused of conspiring to violate foreign agent laws. The consulate operated until July 15, 1941, five days past a city-imposed deadline to vacate.

After Germany's consular officials were expelled from the United States, the mansion stood empty, watched over by a caretaker named Captain Fred C. Mensing. On September 24, 1947, the Alien Property Custodian seized the building. It was transferred to the U.S. Attorney General and sold at auction in 1950, returning briefly to use as a private residence before the California Historical Society occupied it from 1956 to 1991. Today, the mansion stands on Jackson Street, its 30 rooms, built-in steel vault (expanded and fortified by the Nazis with an alarm system), and one of San Francisco's first residential elevators all intact. It is listed on the National Register of Historic Places and designated a San Francisco Landmark. Some say it is haunted. Given what transpired within its walls, it would be strange if it were not.