I have not been upstate in a long time but I would assume Woodstock is still very art and music oriented, and not far from NYC in the Catskills. But, it is coooolllllldddd outside! So not the best time for a trip north unless they love hanging out by the fireplace.
Another suggestion, if we are going to ignore the cold, is Washington DC. Four hour train ride, less if you pay for the fast train, and the museums are fantastic and free!
Lastly FL is usually inexpensive in January and early February. President’s day in FL is the kick off to massive amounts of vacationers through Easter, so prices are generally very reasonable, both air fare and hotels after new years and before presidents day. If FL interests you I can give you more details. I will say this, for more of a guarantee of beach weather stay south of Sarasota and Vero. It is a good time for Disney, no lines, but it can be chilly in Orlando that time of year, hit or miss.