Henry William Portman began developing this land in Marylebone in 1765, and the construction of the square's terraced houses extended until 1784. The Portman Estate — which still owns the private communal gardens at the centre — leased the properties on long terms with ground rents, a model of development that kept the square's character consistent through generations of different occupants. The western end of Wigmore Street meets the square here, connecting it eastward toward Cavendish Square, so the address came with geographic convenience as well as social cachet. An infantry barracks — the Portman Square Barracks — occupied ground between Portman and Orchard Streets until its demolition around 1860. Its temporary presence speaks to the military housing pressures of the era.

The square's north side contains some of its most distinguished architecture. Nos. 11–15, built between 1773 and 1776 by architect James Wyatt in collaboration with his brother Samuel Wyatt, were among the first houses in Britain to use Coade stone, an artificial stone manufactured to precise specifications that proved remarkably durable and weather-resistant. They were demolished in the 20th century, a loss felt in architectural circles. No. 20, known as Home House, was built by Robert Adam between 1773 and 1777 for Elizabeth, Countess of Home; it later housed the Courtauld Institute of Art before becoming a private members' club. No. 22, Montagu House, was built in the northwest corner by architect James Stuart between 1777 and 1781 for Elizabeth Montagu, the celebrated literary hostess and founder of the Bluestocking Society. It was destroyed during the Blitz by an incendiary bomb. No. 30, now the Churchill Hotel, was once the home of George Keppel — husband of Alice Keppel, who was the mistress of King Edward VII.

The list of early occupants reads like a roll call of Georgian aristocracy. Alexander Hamilton, 10th Duke of Hamilton, leased here; so did Sir Brook Bridges, 3rd Baronet; Henry Pelham-Clinton, 4th Duke of Newcastle-under-Lyne; George Keppel, 6th Earl of Albemarle; Sir Charles Asgill, 1st Baronet; and William Henry Percy. These were not occasional visitors. The square was built for people who expected to stay — who leased in toto, bringing their households, their servants, and their social networks with them. Alexander Duff, 1st Duke of Fife, took a lease on No. 15 in the weeks before his marriage to Princess Louise; the princess then lived there for more than four decades. That scale of commitment was the point. Portman Square was designed to hold its residents' lives.

At the eastern end of the garden stands the Hamilton Memorial Drinking Fountain, provided by Mariana Augusta in memory of her late husband Sir John James Hamilton, 2nd Baronet, who had briefly served as MP for Sudbury. The fountain is Grade II listed. It is the kind of civic gesture that polite Georgian society made as a matter of course — public, dignified, modest in scale. The square today is quieter than it once was, surrounded by the offices and hotels that have gradually replaced the residential aristocratic life it was built to contain. But the private gardens at its centre remain, still owned by the Portman Estate, still closed to those who do not have a key.