Montpelier's roots reach back to 1723, when James Madison's grandfather Ambrose received a land patent in the Virginia Piedmont and moved his family there in 1732, naming the place Mount Pleasant. Ambrose died six months later, poisoned by three enslaved Africans according to court records. The property passed through generations, and by the time James Madison returned in 1797 with his new wife Dolley, a young widow with a child, he began transforming the house in earnest. He added a thirty-foot extension and a Tuscan portico, and later, between 1809 and 1812, expanded again with a large drawing room and one-story wings at each end. After his second presidential term ended in 1817, Madison retired here permanently, and the 22-room mansion became the center of his intellectual world. The name itself carries a touch of continental aspiration. First recorded in a 1781 letter, Madison preferred the French spelling Montpellier, after the famous resort city in southern France.

About 100 enslaved people sustained Montpelier during Madison's ownership. They worked the fields, cooked, cleaned, operated the mill and forge, and practiced skilled trades in carpentry and wheelwrighting. Paul Jennings, born at the estate in 1799, became Madison's personal attendant and accompanied him to Washington. Senator Daniel Webster purchased Jennings from the widowed Dolley Madison in 1845 and granted him his freedom. Jennings then became a homeowner in Washington, D.C., working at the federal Pension Bureau. In 1848, he helped plan the Pearl escape, in which 77 enslaved people from the Washington area boarded a schooner to sail up the Chesapeake Bay toward a free state. They were captured and most were sold to the Deep South. His 1865 book, A Colored Man's Reminiscences of James Madison, remains the earliest known White House memoir. Archaeological research has also illuminated the life of Catherine Taylor, born enslaved at Montpelier around 1820, who with her husband Ralph petitioned for freedom after years of separation and legal limbo, finally achieving it in 1853.

In 1901, William and Annie Rogers duPont purchased the estate and remade it in their own image. Horse enthusiasts deeply involved in Thoroughbred racing, the duPonts built barns and stables, covered the original brick in stucco for a lighter look, and more than doubled the number of rooms to 55. When William died in 1928, daughter Marion inherited Montpelier. She made only one change to the house itself, remodeling her parents' music room in Art Deco style with laminated plywood, chrome, glass block, and plate glass mirrors. A weather vane installed on the ceiling tracked wind direction for fox hunting. In 1934, Marion and her brother William founded the Montpelier Hunt Races, a steeplechase using natural hedges as jumps that still runs annually on the first Saturday of November. When Marion duPont Scott died in 1983, she bequeathed the property and a $10 million endowment to the National Trust for Historic Preservation.

A $25 million restoration completed on Constitution Day 2008 stripped away the duPont additions and returned the mansion to its 1820 appearance, revealing original brick beneath the stucco and rebuilding interiors with horsehair plaster and linseed oil paint. The house shrank back to less than half its expanded size. But the most consequential work happened underground. In the cellars of the Madison house, the exhibition The Mere Distinction of Colour opened in 2017, funded by philanthropist David Rubenstein. The south cellar tells the Montpelier slavery story with the names of every person known to have been enslaved there. The north cellar confronts the national economics of slavery. Living descendants of the enslaved community guided the exhibition's creation. In 2021, the Montpelier Foundation approved bylaws to share governance with the Montpelier Descendants Committee. By May 2022, descendants held 14 of 25 board seats, including the chair, making Montpelier one of the first major historic plantations where descendants of enslaved people hold governing authority.

Beyond the mansion, the property holds the Montpelier Forest, designated a National Natural Landmark in 1987. This old-growth Piedmont forest of tulip poplar, spicebush, oak, and hickory contains trees up to 300 years old, some predating the Madison family's arrival. Over eight miles of walking trails wind through horse pastures, wildflower meadows, and forests. The steeplechase track, one of the few in the country using traditional hedgerow jumps, hosts seven races each November. Visitors can watch from the rail. The property also draws crowds for a wine festival showcasing Virginia's vineyards and a Fall Fiber Festival in October, complete with sheep shearing, craft demonstrations, and sheepdog trials. From the second floor of the mansion, the Blue Ridge Mountains stretch across the western horizon, the same view Madison contemplated while drafting arguments that would become the Bill of Rights.