Lee's Army of Northern Virginia had spent the previous week mauling John Pope's Army of Virginia at the Second Battle of Bull Run, then failed to deliver a knockout blow at Chantilly in a thunderstorm on September 1. Pursuing the Federals further toward the Washington fortifications was, Lee decided, a bad idea. The terrain favored the defender and his men were exhausted. He chose instead to swing northwest into Loudoun County to forage, rest, and reorganize before crossing into Maryland. The plan was bold and it required clean roads. Loudoun was held by Federal cavalry working out of the garrison at Harpers Ferry, including Major Henry Cole's 1st Potomac Home Brigade Maryland Cavalry and the Loudoun Rangers - a small Unionist outfit raised from local German Quaker farmers near Waterford under Captain Samuel C. Means. J.E.B. Stuart sent the 2nd Virginia Cavalry under Tom Munford ahead to clear them out. Munford had 163 troopers.

On the morning of September 2, Munford reached the eastern outskirts of Leesburg and split his command in two. He sent Captain Jesse Irvine, Jr. with one squadron straight up the Leesburg Pike - today's Route 7 - and into town, while he took the rest of the regiment north off the pike toward Edward's Ferry. Irvine clattered into Leesburg and found Means and his Loudoun Rangers guarding the courthouse on Market Street. The Rangers had been roughed up only a few days earlier by Elijah White's Confederate partisans at Waterford, and they had no stomach for another fight. They retreated north up King Street - today's Route 15 - leaving four wounded behind. Irvine pursued. The Rangers fell back on Cole's Maryland Cavalry, who had taken a dismounted position about a mile north of the courthouse near Big Spring, where Tutts Lane now branches off. Cole's troopers braced for what they thought would be a frontal cavalry charge. They did not expect Munford.

Munford had been working his way north along the river the whole time, hidden from view by the rolling ground. He came up Smart's Mill Lane - the access road behind Cole's position that nobody had thought to picket - and struck the Maryland Cavalry from the rear. The volleys hit men on foot, men trying to find their bridles, men still arguing with the dismounted skirmishers facing Irvine. Cole's command came apart. Of the troopers who managed to mount, a handful fought briefly before retreating northwest toward Catoctin Mountain and the gap that led to Waterford. Munford chased them two miles, drove them through the gap into the Loudoun Valley beyond, and turned back. Behind him, Leesburg lay open. By the time he counted his losses, the math was favorable - two Confederates killed and five wounded, against six Federals killed, twenty-seven wounded, and eleven captured. A disproportionate share of the Federal casualties were officers, which suggests the rifle fire had been deliberate.

On September 4, Robert E. Lee rode into Leesburg with the Army of Northern Virginia. The town was Confederate in sympathy and welcomed him without resistance. Two days after that, on September 6, his vanguard crossed the Potomac at White's Ford a few miles north. The Maryland Campaign had begun. The campaign would end at Sharpsburg on September 17 in the bloodiest single day in American history, with more than 22,000 casualties between both armies. The skirmish at Mile Hill never gets the attention that Antietam does, but without it the campaign might have looked very different. The battlefield itself is mostly gone. Cole's initial position was buried under the U.S. Route 15 Bypass and two public schools - Smart's Mill Middle and Frances Hazel Reid Elementary. Smart's Mill Lane, the unguarded approach that decided the fight, is now a residential subdivision. A fragment of the old road survives within it. Civil War Trails signs near Tutts Lane mark the rough location, and the open ground of Morven Park and Ida Lee Park preserves something of the setting where Munford's troopers came over the rise and found Cole's men still looking the wrong way.