Sailing is not physically demanding in some of the ways that football and basketball are—there’s no running, jumping, smashing into other bodies or throwing a ball involved—but when done right it has physical demands of its own that can be (should be) pretty strenuous. Those involve keeping the boat in good trim, so that it doesn’t heel too much (or the wrong way). That requires a crew that’s constantly ‘hiking out’ to windward (good abs and legwork required) and maintaining control of sheets (the ropes that control sail movement). It also requires a fine feel for the angle of attack to the wind, waves and current to maintain the best speed over the ground (that is, over the ground under the water). That ‘feel’ comes from long experience and time on the water. (I’m often amazed by what new sailors can’t feel when they get on the water, even after several experiences on the boat.)
Golf and bowling are often derided as pansy sports (or sports for drunks), but to be good at either of those requires another kind of athleticism: the ability to do well and do it consistently. That is, a bowler facing a spare pickup has to know exactly how his ball will move down the alley and precisely where he needs to strike one pin to knock down the other/s as well. Golfers have to know which club to use to place the ball ‘just so’, and if you ever watch the hours that good golfers spend on a driving range hitting a bucket of balls into a relatively small area time after time you’ll realize that there are very few “lucky shots” by good golfers.
Since all of the sports that I’ve mentioned so far involve good body control and consistency, maybe that’s a way to determine who are athletes. Even poker and chess might qualify, since a good poker player needs to learn how to control his ‘tells’ (the giveaway signs that he has a good or bad hand), and a chess player needs to stay relatively still and patient, yet concentrated on the game.