Amputation?
In all seriousness, there continues to be significant debate about whether a real association exists between weather and arthritis symptoms. Some people swear by this however (i.e. the rain or cold or low pressure etc. excacerbate symptoms).
In terms of what to do, you should start with a thorough medical evaluation to determine what type (if any) of arthritis you have as treatments vary depending on the diagnosis. In the meantime, you can try starting with some tylenol to help alleviate the pain. I know that some people believe in the utility of chondroitin sulfate, but there is little to no data supporting its use, including a large study published in 2006 in the New England Journal of Medicine.