Polk was born around 1732 in Cumberland County, Pennsylvania, to William and Margaret Taylor Polk, Scots-Irish farmers who had crossed the Atlantic a generation earlier. In 1753, at about twenty-one, he moved south to Anson County, North Carolina, joining the wave of Scots-Irish settlers pushing down the Great Wagon Road into the Carolina backcountry. He married Susanna Spratt in 1755, and they raised eight children together. The Polk homeplace sat at the intersection of the Indian Trading Path and a smaller trail, on land just inside Catawba territory. That crossroads would become Charlotte. Polk was a commissioner of the new town from its founding, partly through the influence of land speculator Henry McCulloh, with whom Polk had a complicated relationship: he led settlers in the War of Sugar Creek in 1765 to drive McCulloh's surveyors off contested land, then accepted McCulloh's patronage when the dust settled.

By the spring of 1775, Mecklenburg County's Scots-Irish Presbyterians were among the most aggressive Patriots in the South. On May 31, 1775, Polk was among the local officials who adopted the Mecklenburg Resolves, declaring laws enforced by the Crown null and void and calling for a wholesale reorganization of colonial government. The Resolves predated the Declaration of Independence by more than a year. (A separate, more famous document called the Mecklenburg Declaration of Independence, said to predate Jefferson's by a year, remains disputed by historians, but the Resolves themselves are documented.) Polk was promptly elected to the Third North Carolina Provincial Congress and commissioned a colonel of Patriot militia. He marched into the South Carolina Upcountry that winter as part of the Snow Campaign against Loyalist recruiters.

In 1776 Polk was named colonel of the 4th North Carolina Regiment of the Continental Line. His regiment marched north in early 1777 and fought at the Battle of Brandywine on September 11, where Washington's army was outflanked by Howe and forced to retreat toward Philadelphia. Days later, Polk drew the assignment of escorting the city's bells out of harm's way. The wagons rolled to Allentown, the bells were hidden under the floorboards of the Zion Reformed Church, and Polk pressed on to Valley Forge with the main army. He spent the brutal winter of 1777-78 in those huts. By the following summer he was back in North Carolina to recruit, but resigned his commission on June 26, 1778, bitter at being passed over for promotion to brigadier general after the death of Francis Nash.

Polk's military career was not finished. In mid-1780, with the British rolling through the Carolinas, he accepted appointment as commissary general of purchases for the Continental Army in the southern theater. He used his own credit and personal assets to keep Patriot soldiers fed, a practice that nearly ruined him financially. When Nathanael Greene took command of the southern army in late 1780, he spent his first night in camp in long conversation with Polk, learning the region's roads, rivers, and resources from a man who had walked them his entire adult life. Greene later nominated Polk for brigadier general; the North Carolina General Assembly refused the rank, offering 'colonel commandant' instead. Polk, exhausted and aging, declined.

After the war Polk served on the North Carolina Council of State in 1783 and 1784. In 1786 the General Assembly elected him to the Congress of the Confederation, but he never attended a session. The most-told story of his later years comes from May 1791, when President George Washington toured the southern states and slept under Polk's roof on his stop in Charlotte. Polk died at his Charlotte home on January 25, 1794, and was buried in what is now the Old Settlers' Cemetery in the heart of the city he had helped found. His brother Ezekiel was the grandfather of James K. Polk, the eleventh President of the United States. The two strands of the Polk family that converged at Mecklenburg's crossroads would, two generations later, produce a man who carried the nation to the Pacific.