It depends on what you’re used to and also what the combination of air and water temperature is like.
For example, Canadians and New Englanders go to Myrtle Beach, SC, in March and have a great time in the water, and the natives think that we’re nuts. (I myself haven’t spent ‘a lot of time’ in that water at that time of year, but I can handle it for awhile if the air is warm enough to dry in.)
So if the water is cool but the air is warm, it can work. Likewise if you’re going to a place that has a hot tub next to a pool, for example, then the pool can be a nice place to chill after thorough warming in the hot tub, or vice versa.
My own experience is that if you have either warm air or warm water to get into, then I can handle significant coolness in the ‘other’ medium. But if both are cool, then I’m going to stay in the room or in the car, and if both are overly warm, then you went to the wrong place on vacation, or at the wrong time.