The school building was constructed in 1902 by Charles D. Ogden, a Harrison County contractor, on what was then Water Street in Clarksburg's African-American neighborhood. It opened in 1903 as the Water Street Colored School - the only public secondary school in north-central West Virginia that admitted Black students. By the mid-1920s the building had outgrown its original capacity. In 1929 the school was substantially expanded to include a gymnasium, a swimming pool, a large library, an auditorium with 825 seats, additional classrooms, a manual arts workshop, and a first-class home economics department. The expansion gave Kelly Miller facilities that compared favorably with the better white high schools in the region - a deliberate effort by the African-American community, and by some sympathetic white officials, to provide the kind of physical plant that would not signal second-class education to either students or visitors.

Like most segregated Black schools of the period, Kelly Miller served as much more than a school. The auditorium hosted community meetings, lectures, concerts, and religious services. The gymnasium was the venue for the principal Black social events in Clarksburg. The home economics rooms and the manual arts workshop offered classes for adults in the evenings. The Yellow Jackets - the school sports teams, in black and orange - represented the African-American community in regional competition against other Black schools across the state. For Clarksburg's Black families, Kelly Miller was the institution they could identify with most strongly: the place their children were educated, the place where their community gathered, and the place that held the trophies and graduation photographs that documented generations of upward effort against considerable headwinds.

Kelly Miller himself was born in Winnsboro, South Carolina, in 1863, the year of the Emancipation Proclamation. He graduated from Howard University in 1886 and became the first African-American admitted to Johns Hopkins University, where he studied mathematics, physics, and astronomy from 1887 to 1889 - a remarkable achievement at an institution that had been founded only a decade earlier and that did not admit other Black graduate students for many years after Miller's brief enrollment. He returned to Howard in 1890 as professor of mathematics, introduced sociology into the curriculum in 1895, and served as professor of sociology from then until 1934. As dean of the College of Arts and Sciences, he modernized Howard's curriculum and built up the natural and social sciences. He graduated from Howard's School of Law in 1903. His weekly newspaper column, syndicated in Black newspapers across the country for decades, made him one of the most widely read commentators on race in America. He assisted W. E. B. Du Bois in editing The Crisis, the NAACP's principal journal, and authored books including Out of the House of Bondage. His 1920 lecture in Clarksburg was one of hundreds he gave across the country during his long career.

Two principals defined the school's character. Duncan Huey Kyle, the founding principal in 1903, was one of a group of African-American educators trained in Alabama and sent out across the South and lower Midwest in the late nineteenth and early twentieth centuries to lead the new generation of segregated Black schools. He served at Kelly Miller until 1919. He was succeeded by E. B. Saunders, who served as principal from 1919 until the school closed in 1956 - a remarkable thirty-seven-year tenure that spanned the Roaring Twenties, the Great Depression, World War II, and the early Civil Rights movement. Saunders was deeply beloved in the Clarksburg community. After Kelly Miller closed, he became principal at Linden Grade School, where he finished his career. In the 1990s the city of Clarksburg renamed the portion of Water Street where the school stands EB Saunders Way in his honor.

The Supreme Court's 1954 Brown v. Board of Education decision required the desegregation of public schools, and Kelly Miller closed in 1956 as Black students were integrated into the previously white schools of Clarksburg. The building was used for various educational purposes over the following decades and from 1982 served as the central offices of the Harrison County Board of Education. In recent years the board has been planning to move to the former Gore Middle School, leaving Kelly Miller's future uncertain - until the West Virginia Black Heritage Festival, which holds its annual September gathering in uptown Clarksburg, acquired the building and began converting it into a community center. The intent is to restore its historic role as the cultural anchor of African-American Clarksburg, with exhibits on the school's history, the principals, the students, and Kelly Miller himself - the South Carolina-born scholar whose 1920 visit to Clarksburg gave the school the name it has carried for more than a century.