A mill stood on this site before the Tudors. The present building dates from 1542, originally known as Bedell's or Biddle's Mill after one of its early owners, Samuel Biddle. A 1727 description called it High Wheel Mill. By 1755, the lease had passed to Matthew Boulton, the Birmingham industrialist and central figure of the Lunar Society. Boulton converted some of the machinery for metalworking, using the same waterwheels that ground grain to roll sheet brass and sharpen tools and draw wire. The mill, in other words, was a small node in the Industrial Revolution before James Watt had even finished his steam engine. The wheel that drove the millstones drove drop-hammers, then drew wire, then ground grain again as Boulton's interests moved elsewhere. Industrial archaeology is rarely this layered.

In 1852, a single-cylinder table engine of 16 horsepower was installed to supplement the waterwheel. Water remained the primary source of power, but the engine ensured the mill could keep running when the river ran low. The original engine was removed at some point. The one now on display, a similar size and type, was installed during the 1975 restoration. It came from Smith Kendon Ltd, a sweet manufacturer with factories in England and Messina in Italy, where it powered confectionery production until 1948. After the Second World War, the mill itself fell into dereliction. A community campaign saved it from demolition. Restoration was completed in 1969, and the mill reopened as a museum operated by what is now the Birmingham Museums Trust.

Mabel Tolkien moved her two sons to a cottage at 5 Gracewell, on what is now Wake Green Road, in 1896. Her husband had died in South Africa earlier that year. The hamlet of Sarehole was then rural Worcestershire, surrounded by fields and farms and one working mill. Ronald, the elder of the two boys, was a small bookish child. He would later say that the years between four and eight were the most formative of his life. In a 1966 Guardian interview with the journalist John Ezard, Tolkien identified Sarehole Mill specifically as the model for the Mill at Hobbiton. The miller's son, who chased Tolkien and his brother away when they trespassed, became the White Ogre of the boys' games. Birmingham annexed Sarehole in 1911. By then Tolkien had been gone for a decade, but the wheel had already begun turning in him.

In April 2012 the millpond was drained so the sluice gates could be repaired, and the following winter the silted pond was dredged for the first time in decades. Flood damage in 2019 stopped flour production. By February 2020, the Victorian bakery on site had been restored, with a working modern bakery installed alongside it. Then the COVID-19 pandemic closed the mill, along with every other location managed by the Birmingham Museums Trust. The mill has since reopened to visitors. The Millers Tea Room sells bread baked on the premises. Guided tours, including the Origins of Middle Earth route that walks through the Tolkien sites in the surrounding area, run regularly. Sarehole is one of only two working water mills left in Birmingham, the other being New Hall Mill in Walmley, and the only one open as a museum.

The mill sits within the Shire Country Park, a strip of green that follows the River Cole through south Birmingham. Moseley Bog, where Tolkien wandered the woods, is fifteen minutes' walk upstream. The millpond, calm enough to mirror the building most days, is fringed with willows. Mallards, moorhens, and the occasional kingfisher work the bank. Inside, a brass plaque marks the entrance, and the smell of newly ground wheat fills the lower floor when the wheel is running. Children look up at the gears with the same expression a small boy from Bloemfontein wore in 1896. Tolkien did not invent the Shire. He remembered it. Birmingham, the city that swallowed Sarehole in 1911 and nearly demolished the mill in 1969, has the good sense now to keep grinding flour.