Cool question, to which I have absolutely no answer. Disregarding learning disabilities and left brain/right brain stuff, which not everyone knows about…I’d say that it is indeed a clear observation; lots of folks think being bad at math isn’t a problem, but reading/writing problems makes you a savage.
Instead of looking at math, I’m gonna look at reading. Reading and writing are extremely valuables things to know, no matter what you do…one could say the same for math as well, but in most every day things that require math, it only requires the basics, and even someone who doesn’t know their multiplication table can get away with it using tools like computers and calculators. I mean, cash registers pretty much do all the work for you. May not have been this way before, but it is now. More complex mathematics aren’t present everywhere. Now again, the same can be said of reading and writing, but I think its basics are much wider in scope of everyday use, plus there aren’t many secondary tools that can save your ass if you can’t properly read or write something.
I do believe that math has its own personal taboo as generally being either pretty hard, or really boring. So people agree on that. But reading and writing is like breathing for most, so illiterate folk get the hack. Or some dumbass societal factor therelike. (izzat even a wurd)
Now I’m not saying math is useless, bloody far from it, and I ain’t saying that reading and writing can escape on slang and cultural habits; but besides the technical stuff that separate both, there is some social factor that must play a part in how people perceive both subjects when it comes to being good/bad at them.