The college takes its name from John Randolph of Roanoke, the Virginia statesman known for his caustic oratory and erratic genius, and Nathaniel Macon, the long-serving North Carolina senator and Speaker of the U.S. House of Representatives. Neither man founded the school. The actual founders were Methodists Hezekiah G. Leigh and John Early, along with Staten Island native Gabriel Poillon Disosway. They chose Boydton — near the Virginia-North Carolina border in Mecklenburg County — for its accessibility to Methodist communities in both states. The first campus opened in 1832. In 1847 the college established a relationship with Hampden-Sydney College, leading to the formation of the Randolph-Macon Medical School, which closed in 1851. The college's president William A. Smith delivered a defense of slavery in lectures published in 1856 and 1857 — part of the proslavery intellectual project of the antebellum South. That history is part of the institutional record. The college today acknowledges it explicitly.

The Civil War broke the railroad connection to Boydton, and after the war the trustees decided that rebuilding in place was not feasible. In 1868 the college moved to Ashland — a railroad town about 15 miles north of Richmond, on the Richmond, Fredericksburg, and Potomac line. The original Boydton campus became the home of the Boydton Academic and Bible Institute, a Christian school for African Americans that operated from 1878 until 1935. At the new Ashland campus, Washington-Franklin Hall went up in 1872 — the first brick building in Ashland, constructed with funds raised by students. It still stands and now houses the history department. Pace-Armistead Hall (1876, renovated 1997) was originally the chemistry building; today it is the studio art building. The original Duncan Methodist Church (1879) became classrooms and offices for music and arts. All three buildings are on the National Register of Historic Places as the "Historic Campus."

For a small liberal arts college, Randolph-Macon accumulated an unusual number of "firsts." It became the first college south of the Mason-Dixon line to require physical education for graduation. The old gym built in 1887 was the first structure in the South built solely for instruction in physical education. The college is considered the first in the South to offer English as a full discipline and to develop biology as a distinct field of study. Its computer science department, established in the 1960s, was among the first at any liberal arts college in the United States — at a time when many academics still believed computer science belonged to commercial trade schools rather than serious universities. In 1923 the college received the Zeta of Virginia chapter of Phi Beta Kappa, the nation's oldest academic honor society. Chi Beta Phi, the national science honorary society, was founded at Randolph-Macon in 1916. The college became coeducational in 1971 with the enrollment of 50 women and the first full-time female faculty member.

The Randolph-Macon vs. Hampden-Sydney football rivalry dates to 1893 and has been called the oldest small-school rivalry in the Southern United States. Both schools are NCAA Division III. The matchup is known simply as "The Game." It has expanded over the years to encompass all sports, and the men's basketball series has drawn national attention as both programs have become powerhouses in the Division III ranks. The Yellow Jackets compete in 20 intercollegiate varsity sports in the Old Dominion Athletic Conference, where they have been members since the 1976-77 academic year. The 2022 basketball national championship was the college's first. Coach Nathan Davis took over the program in 2009 and built it methodically; the 2022 squad ran through the tournament with the same blend of discipline and shooting that has defined Davis-era Yellow Jacket basketball. The Brock Recreation and Athletic Center anchors the modern athletic facilities, alongside new football and baseball stadiums completed in the 2010s capital campaign.

Randolph-Macon today enrolls more than 1,800 students on a 116-acre campus with over 65 academic, administrative, athletic, and residential buildings. The main north-south CSX railroad line — the spine of the east coast — still runs straight through the campus, with most buildings to the east and a handful of college offices, special-interest houses, and athletic fields to the west. The Ashland train station, served by Amtrak, sits directly across from the southern entrance to campus. In 2018 the college completed a capital campaign that exceeded its $100 million goal, funding two new residence halls, the rebuilt Brock Commons student center (2013), and renovations across the science and library buildings. Newer additions include Payne Hall (2020, nursing), Duke Hall (2023, physician assistants studies), and the Spotswood Village apartments (2025) west of the tracks. The college continues to offer three undergraduate degrees and added master's-level programs in 2023. The campus is small enough to walk across in under fifteen minutes. The trains, every few hours, still cut it in half.