Why is it a tough thing to answer? Are you so old that you forget? In that case I would agree that it could be a tough thing, especially if you’re shy about admitting to your failing memory or you have to refer to an index card that you carry around with your name, address, telephone number – and birthdate – printed in large letters (because of your also failing eyesight).
However, the best response is the classic deadpan – nothing. If the question is repeated, then a wide-eyed, “Why, whatever in the world has possessed you to ask THAT?” and if the question is repeated AGAIN, by someone particularly dense, rude or uncaring, then “It’s none of your goddamn business how old I am. How rude are YOU?”