Before it had a name, the route was known as the road to Reading or the way from Colnbrook — a practical description of a road heading west out of London. It has been a main thoroughfare since at least medieval times, predating London's growth beyond its walls. By 1663, the street had been renamed Portugal Street in honour of Catherine of Braganza, wife of Charles II; it grew in importance when the road from Charing Cross to Hyde Park Corner was closed in 1668 to allow the creation of Green Park. The aristocracy moved in quickly. Clarendon House and Burlington House were both built on the northern side in 1664. Burlington House has since become home to the Royal Academy of Arts, the Geological Society, the Linnean Society, and the Royal Astronomical Society, making its courtyard one of the densest concentrations of learned institutions in Britain.

By the 19th century, Piccadilly had become a measure of wealth. Nathan Mayer Rothschild moved his banking premises to No. 107 in 1825. Other members of the Rothschild family built mansions nearby with ballrooms and marble staircases; the western end of the street was colloquially known as Rothschild Row. Albert, Duke of York — later King George VI — was living at No. 145 Piccadilly at the time of his accession in 1936. Hugh Mason and William Fortnum had started the Fortnum and Mason partnership on Piccadilly in 1705, initially selling recycled candles from Buckingham Palace; by 1788, the store stocked lobsters, Scotch eggs, and exotic fruits. Hatchards bookshop opened at No. 173 in 1797, moved to its current location by 1801, and survives today as the oldest bookshop in Britain. St James's Hall, built between 1857 and 1858, hosted performances by Dvořák, Grieg, and Tchaikovsky before being demolished in 1905. Charles Dickens read from his novels there.

Piccadilly's 20th century was not straightforwardly grand. Jazz trumpeter Dizzy Reece recalled people queuing outside a Piccadilly branch of Boots for heroin pills in the late 1940s. By the 1960s, the street and its surrounding area were notorious as the centre of London's illegal drug trade, where heroin and cocaine could be purchased from chemists operating outside the law. In 1968, squatters occupied No. 144, taking advantage of legislation that allowed disused buildings to be used as emergency shelter. The radical squatting movement this inspired collapsed when drug dealers and Hells Angels moved in; an eviction took place on 21 September 1969. The episode eventually resulted in the licensing of legitimate squatting organisations. Meanwhile, the gentlemen's clubs that had defined the street's social character through the late 19th and early 20th centuries largely disappeared. Of that long tradition, only the Cavalry and Guards Club and the Royal Air Force Club remain.

The street appears in more novels, plays, songs, and films than almost any other in London. E. W. Hornung's gentleman thief Raffles lives at the Albany — a residential conversion at the western end that also housed Prime Ministers William Ewart Gladstone and Edward Heath. Jack Worthing in Oscar Wilde's The Importance of Being Earnest gives his address there too. Bram Stoker sends Dracula to a house in Piccadilly before the climax of the novel. Dorothy Sayers' Lord Peter Wimsey lives at 110A Piccadilly. P. G. Wodehouse used the street as the natural habitat of wealthy, idle bachelors. It appears on the Monopoly board, in military songs, in Gilbert and Sullivan. The Piccadilly line runs underneath it. The street is, in some ways, less a road than an idea — a place where London performs its own mythology, and has been doing so for centuries.