The land itself holds layers of history older than the building. This was once part of the estate of British Army colonel Roger Morris, who acquired acreage in Upper Manhattan in 1765 and built the Morris-Jumel Mansion at the top of Coogan's Bluff, a cliff overlooking the Harlem River. The Morris family fled in 1775, and the estate served as headquarters for both the Continental Army and the British during the Revolutionary War. The Jumel family bought the property in 1810. By the late nineteenth century, the estate had been subdivided, and in 1908, contractor Albert J. Schwarzler purchased ten lots on the western side of Edgecombe Avenue. He hired Schwartz and Gross to design a twelve-story apartment building on the northern half of his parcel, filing plans in May 1914. The Department of Buildings initially disputed the building's height, claiming fifteen stories and 156 feet. The architects countered that the basement was above ground only because of the slope, and they prevailed.

When 555 Edgecombe opened in January 1916, its 105 apartments housed entirely white tenants, most American-born, with some immigrants from European countries and Canada. The shift came swiftly. Telephone directories tell the story in cold numbers: in 1938, eighty-three families were listed. By 1939, only seventeen remained. By 1940, none of the original white tenants were left. Schwarzler had let their leases lapse as the surrounding neighborhood became predominantly Black. Sixty-seven new tenants, all of whom were likely Black, appeared in the directories between 1939 and 1940. What was displacement for some became opportunity for others. The building, along with nearby 409 Edgecombe Avenue, quickly earned a reputation as one of Sugar Hill's "class houses," home to well-off African Americans who had been shut out of housing elsewhere in Manhattan.

The list of residents reads like a catalog of twentieth-century Black achievement. Paul Robeson, the actor, singer, and activist whose name the building now officially bears, lived here. Count Basie, whose band defined the swing era, had an apartment. Coleman Hawkins, the saxophonist who essentially invented the tenor saxophone as a jazz solo instrument, was a tenant. So were singer-songwriter Timmie Rogers, bandleader Andy Kirk, and psychologist Kenneth Clark, whose research on the psychological damage of segregation was cited by the Supreme Court in its Brown v. Board of Education decision. In 2009, the New York Daily News observed that the Triple Nickel "drew more African-American celebrities than any building ever could today." The intersection outside was co-named Paul Robeson Boulevard and Count Basie Place that same year. The building was declared a National Historic Landmark under the name Paul Robeson Residence in 1976, and designated a New York City landmark in 1993.

Ownership of the building changed hands with remarkable frequency, and nearly every transition brought conflict. After Schwarzler died in 1941, the estate sold both 545 and 555 Edgecombe to a buyer whose manager, Cecil Carter, promptly cut services: the doorman was fired, elevator operators were dismissed, and residents organized a thirty-two-person rent strike. The owners relented and fired Carter. In 1947, Daddy Grace, the charismatic founder of the United House of Prayer For All People, purchased both buildings. When workers went on strike in 1949, Grace reportedly had his followers operate the elevators. After Grace's death in 1960, a complicated legal battle erupted as his followers tried to prevent the sale of the property. Matthew Golson bought the building in December 1960 and his family held it for over six decades, until the Harkham family purchased it in April 2022 for $26.7 million, a fraction of the $65 million asking price Golson had set five years earlier.

In the 1990s, a resident of 555 Edgecombe began hosting jazz concerts in her apartment, performances that became popular gathering points for local musicians and a quiet continuation of the building's musical heritage. The tradition spoke to something essential about the Triple Nickel: it was never just an address. It was a social ecosystem, a place where proximity bred collaboration and where a shared hallway could lead to a jam session. A short documentary about 409 and 555 Edgecombe Avenue, In the Face of What We Remember, was released in 2018. The title captures the building's enduring significance. Memory is embedded in the brickwork here, in the arched entrance with its ironwork, in the lobby decorated with plaster and marble and classical relief panels, in the terrazzo floors and the steep slope that makes the building seem to grow taller as you approach from below.