The land that became Smithfield was first held under a 1671 grant of about 5,000 acres from Charles II of England to Major Lawrence Smith of Gloucester County. The Algonquian-speaking peoples who fished the Rappahannock had lived on and around the property for generations before that. Smith built a small fort in 1676 near what is now the country club's 10th tee. The settlement effort failed, the grant was broken into parcels, and in 1730 a portion was sold to Francis Taliaferro. By the mid-eighteenth century the property had passed to the Brooke family. Richard Brooke raised four sons in the house, all of whom served in the Revolutionary War. Laurence sailed with John Paul Jones aboard the Bonhomme Richard as ship's surgeon during the engagement with HMS Serapis. Robert was captured by the British twice, escaped through Scotland and France, and was later elected Governor of Virginia in 1794. The twins Francis and John served as junior officers at sixteen. Francis became a judge in 1804 and held the bench for forty-five years until 1849.

John Pratt of Caroline County bought Smithfield in 1814. The original Brooke dwelling burned down shortly after, and by 1822 Pratt had built the brick mansion that still stands. The Pratt family held the property through the rest of the nineteenth century. During the Battle of Fredericksburg in December 1862, the house found itself directly in the path of the war. Union troops crossed the Rappahannock to the northeast and marched down the Richmond Stage Road toward what is now the country club, planning to swing right and attack the Confederate line on the high ground a mile to the west. The mansion stood between the Union approach and the Confederate ridge. Most of the senior officers in both armies - Lee, Jackson, Longstreet on one side; Burnside, Hooker, Sumner on the other - had clear sight lines to the house. During the fighting it served as a Union hospital. Cannonballs crossed the property in both directions, Confederate artillery firing from the heights to the west, Union artillery firing from the east bank of the river.

The most dramatic moment on Smithfield's ground belonged to John Pelham, the young West Point graduate who commanded the Stuart Horse Artillery. As the Union division of George Meade began its advance toward Jackson's line, Pelham positioned two guns - a Napoleon and a Blakely rifle - at the intersection of what is now Tidewater Trail and Benchmark Road, just south of the Smithfield house. He opened fire on the Union flank. The Blakely was disabled almost immediately by counter-battery fire. Pelham kept the Napoleon firing alone. J.E.B. Stuart sent orders for him to withdraw. Pelham held the position. Lee, watching from Lee's Hill, was reported to have said It is glorious to see such courage in one so young. Pelham finally pulled back only when his ammunition was nearly exhausted and his crew badly thinned. He had delayed Meade's advance by nearly an hour and dragged the entire Union right flank into confusion. The National Park Service has placed a marker at the spot. Pelham himself was killed three months later at Kelly's Ford. He was 24.

In 1905 Captain Reginald Henley Conroy Vance of New York bought Smithfield and added the wings and white columns that give the house its present face. The work was done by architects William Penn Cresson and Nathan C. Wyeth - the same Wyeth who would later design the Oval Office at the White House for President Taft. In 1907 Vance bought a portion of the adjoining Mannsfield estate and renamed his house Mannsfield Hall, transferring the historic name. The original Mannsfield mansion - one of the great houses of pre-war Spotsylvania - had burned to the ground in January 1863 when Confederate troops mistakenly built their cooking fire directly on a hardwood floor. Vance and his wife Clarissa died in the 1922 collapse of the Knickerbocker Theater in Washington under heavy snow. In 1925 a group of Fredericksburg-area citizens bought the property and incorporated as the Mannsfield Hall Country Club. The mansion became the clubhouse and the surrounding land became nine holes of golf, tennis courts, and a swimming pool. The club nearly failed during World War II, was sold at auction in the 1940s to Colonel Richard F. Riddell for $39,000, then bought back in 1946 by former members for $60,000 and reincorporated as the Fredericksburg Country Club. Nine more holes were added in 1961.

The 1822 mansion still anchors the property. From the air the country club reads as a broad green park stretched along the Rappahannock four miles south of downtown Fredericksburg, the brick clubhouse white-columned at its center, the golf course wrapping it on three sides. The 5.8 magnitude Virginia earthquake of August 23, 2011 - the same quake that cracked the Washington Monument - damaged all four of the original chimneys; the Historic Smithfield-Brooke Foundation has worked to fund the repairs. The small pond beside the 9th tee covers what was once the 8th green of the 1925 course. The fort site near the 10th tee is unmarked. Pelham's marker stands at the intersection where his two guns held off a Union division. The mansion behind him, where surgeons worked over wounded soldiers from both sides through the long December afternoon, now hosts weddings.