A. Alex Shuford Jr. paid for the museum's first acquisition in March 1944: Burke Mountain, Vermont, by the National Academy of Design officer Frederick Ballard Williams. The price was $140. Whitener, a landscape painter himself, had artistic connections in New York City - the painters Wilford Conrow and Henry Hobart Nichols among them. Many New York artists summered in the North Carolina mountains, and several took an interest in the small museum, donating work. In 1954, the museum acquired a major group of pieces from the collection of Hobart Nichols, president of the National Academy of Design. The donation included works by Thomas Cole, Asher Brown Durand, John Frederick Kensett, Worthington Whittredge, Edward Henry Potthast, and Robert Lewis Reid - a Hudson River School collection that would be remarkable in any museum, but is genuinely surprising in a Catawba Valley furniture town.

Within a year of its founding, the museum moved into the white clapboard W.W. Bryan house on Third Avenue. It stayed there for fourteen years before relocating to the former Shuford Mills office building on Third Street and First Avenue NW in 1960. By 1984 it had outgrown that space too. Buck Shuford - of the same Shuford family that had funded the first painting - led a $2.6 million campaign to convert the decommissioned Hickory High School building (formerly Claremont High School) into an arts center. In 1986, the renovated structure opened as the Arts and Science Center of Catawba Valley, now known as the SALT Block. The museum has been there ever since, sharing the campus with the Catawba Science Center, the Hickory Choral Society, and the Western Piedmont Symphony.

Whitener's stated aim was for the museum to 'embrace all the arts and crafts of the upper Piedmont region of North Carolina.' The third floor of the building is dedicated to Southern folk art - what was once called Outsider Art, now usually called Southern Self-Taught Art. In 2004, Hickory residents Allen and Barry Huffman donated more than 150 contemporary Southern folk art objects, the largest single acquisition in the museum's history. The artists represented are central to the social history of the rural South: James Harold Jennings, Richard Burnside, Miles Carpenter, Raymond Coins, Abraham Lincoln Criss, Minnie Adkins, Howard Finster, Russell Gillespie, Minnie Reinhardt. Catawba Valley pottery sits beside them - the regional tradition of clay vessels and 'face jugs' with anthropomorphic features, made for generations by potters whose names usually do not appear in art history books.

Hickory Museum of Art first earned American Alliance of Museums accreditation in 1991. It was re-accredited following a 2014 review - one of only nine museums to make that announcement that year. Of the country's 35,000 museums, only about 1,033 are accredited. To get there, a museum has to conduct a year of self-study and then host a two-person inspection team reporting to the AAM's Accreditation Commission. It is the highest national recognition American museums can earn. A wartime art museum founded in a vacant office building, in a furniture town, has carried that distinction for more than three decades. Paul Whitener's hunch that Hickory needed a visual arts center turned out to be correct in ways that even he probably did not anticipate.