The military history of Fort Baker began in 1850, when President Millard Fillmore established the Lime Point Military Reservation on the north side of the Golden Gate, directly across from Fort Point. The land was intended for coastal defense, but the government did not actually acquire it until 1866 due to protracted litigation. Between 1872 and 1876, four barbette batteries were built: at Point Cavallo, on the ridge above Lime Point, and on Gravelly Beach to the west. The fort was renamed in 1897 for Edward Dickinson Baker, a former U.S. Senator from Oregon who had been active in California politics in the 1850s before dying while leading a Union regiment in the Civil War. Baker and his wife are buried at the San Francisco National Cemetery in the Presidio.

During both World Wars, Fort Baker served as the headquarters and training command post for the 91st Division, whose soldiers came from the far western states. The division earned the nickname "The Wild West Division" and developed a distinctive salute tradition rooted in the frontier era. During World War I, the division served in France; in World War II, it fought in Italy. In the 1960s and 1970s, the fort's World War II wooden hospital buildings housed the Sixth U.S. Army Medical Laboratory, which served as the reference laboratory for all medical facilities in the Sixth Army area and conducted virology research. By the late 1980s, the fort had transitioned to an Army Reserve training facility. In 2000, the final uniformed Army elements left Fort Baker, and the 91st Division relocated its headquarters to Parks Reserve Forces Training Area in Dublin, California.

In 2005, the National Park Service reached an agreement with developers to create a retreat and conference center at the fort. Construction began in 2006, renovating thirteen historic lodging buildings and seven commons buildings while adding thirteen new structures. The 142-room Cavallo Point resort, operated by Passport Resorts (which also runs the Post Ranch Inn in Big Sur), features 15,000 square feet of indoor event space, a restaurant called Murray Circle seating 100, and an 11,000-square-foot spa. The property pursued LEED certification for its reuse of existing buildings and sustainable design. Today the fort also houses the Bay Area Discovery Museum and the Institute at the Golden Gate, a sustainability-focused think tank.

Fort Baker sits at a hinge point in the Bay Area's geography. To the south, the Golden Gate Bridge connects it to San Francisco's Presidio. To the north, the Marin Headlands rise into fog-wrapped ridges. Horseshoe Bay curves below, sheltering the Travis Marina and the Presidio Yacht Club. The fort was listed on the National Register of Historic Places in 1973 as part of the Forts Baker, Barry, and Cronkhite historic district. Its essentially intact historic structures and landscapes offer one of the few places in the Bay Area where you can walk through buildings that look much as they did when soldiers drilled on the parade ground. The views of San Francisco Bay are among the best in the region -- which is presumably why the producers of Star Trek chose this spot for the headquarters of an organization that would one day command starships exploring the galaxy.