The railway was the brainchild of Sir Thomas Tyrwhitt, philanthropist and improver, who had founded the small settlement of Princetown on Dartmoor and built the prison there in 1809 to hold prisoners of war. When peace came in 1815 and the prison emptied, Tyrwhitt began searching for a way to make Princetown viable. He wanted to bring lime, sea-sand, timber, and coal up to the moor to enable agriculture, and to carry down granite, peat, and farm produce. A railway, he proposed in 1818 to the Plymouth Chamber of Commerce, would do it. Toll-paying carriers would run their own wagons on the line, just as on a turnpike road, and the whole enterprise would yield investors an 18 per cent return on capital. Subscribers were found. An Act of Parliament passed on 2 July 1819. Tyrwhitt ceremonially laid the first rail on 12 August. None of his predictions about agriculture, returns, or general use would prove correct.

By the time the line opened on 26 September 1823, the financing was a disaster and the traffic mix was nothing like what Tyrwhitt had imagined. One trader dominated everything. Johnson Brothers had acquired the right to extract granite from quarries on Dartmoor and needed to move it down to Plymouth for shipment to London and elsewhere. They negotiated a flat-rate contract of 2 shillings 6 pence per ton, regardless of distance, for 8,000 tons a year. From the railway's opening in September 1823 to July 1825, the receipts told the whole story: Johnson Brothers paid £1,093, and everyone else combined paid £29. The dreamed-of agricultural and general traffic never materialised. The exchequer loans the company had needed to finish construction could not be repaid. Johnson Brothers eventually took a mortgage on the entire line and effectively ran it as their private granite railway, with horse-drawn wagons hauling Dartmoor stone fourteen-plus miles down to the wharves at Sutton Pool.

Rachel Evans, walking past the Dartmoor quarries in 1846, watched three hundred men at work. "Some are boring holes in the flinty rock; others are filling the cavities with powder; some are chipping the rude blocks into shape; others are lifting their ponderous weight by cranes and levers; horses, carts and railway waggons, are in constant employment, to convey away the heavy masses of stone." The granite she described, some blocks twenty feet long, went down the Plymouth and Dartmoor Railway to Plymouth and from there by coastal ship to the capital. London Bridge was built of it. The General Post Office, too. And the new Houses of Parliament, then under construction at Westminster after the great fire of 1834, rose from Dartmoor granite that had ridden a horse-drawn tram-wagon to the sea. The fishbelly cast-iron rails were only three feet long, bolted to stone block sleepers, and the gauge was 4 ft 6 in. None of it was modern even when it was new.

The Earl of Morley, whose land the line crossed, had interests of his own. He held kaolinite deposits at Lee Moor, the soft white china clay essential to porcelain manufacture, and in 1834 a branch was extended east from Marsh Mills to Plympton to bring the clay down. When the South Devon Railway's main line arrived in 1848 and absorbed parts of the P&DR, the Lee Moor traffic kept the original company alive. In 1858 William Phillips, who had taken over Morley's china clay leases, built the Lee Moor Tramway proper, four kilometres of horse-drawn line plunging down from the moor through two counterbalanced inclines: Cann Wood incline at 1 in 11 over 6,600 feet, and Torycombe at 1 in 7 over 2,145 feet. The tramway brought clay down from the high moor to the P&DR for onward shipment, and went on doing it long after horse-drawn tramways had vanished from almost everywhere else in Britain.

When the standard-gauge Princetown Railway opened in 1883, taking over the main line for proper steam locomotives, the original Plymouth and Dartmoor Railway should arguably have ended there. It did not. The Lee Moor Tramway and a small surviving section at the Plymouth end continued exactly as designed, with horses pulling wagons on cast-iron rails set in stone, across the South Devon main line, into the twentieth century. The Battle of the Somme came and went. The Wright brothers flew, then died of old age. Cars replaced carriages on the roads of Devon. And still the horses crossed the railway on their wooden boards, hauling china clay down from the moor at the pace they had set in 1823. The Lee Moor Tramway finally closed in 1960, 137 years after Tyrwhitt laid his ceremonial first rail. The agricultural revolution he had hoped to set off on Dartmoor never came. But for a century and a half, his railway did the one job it could do, and did it patiently, and did it well.