Someone once expalined it like this to me. Imagine there are 10 switches in your brain that control color. 1 and 5 go on for orange, 2 and 6 for green, 2 for blue, 3,5, and 9 for brown. If you are color blind switches 3 and 4 might not work, so all colors that utilize 3 and 4 will not be perceived accurately, or not how most others perceive that color. I figure one of my switches is just a little dim, like a brown out.
I am not a doctor or a medical scientist, so I have no idea if that is really an accurate way to describe it, but it was a nice story. Maybe a doctor will chime in.