Conscience isn’t really the same as being able to tell right from wrong, or moral sensibility. Conscience is one’s self-enforcement of a moral code. Psychopaths are perfectly aware of what’s right and wrong; they know the moral code. What makes them psychopaths is that they lack any internal enforcement mechanism. They do not feel at all bad when they violate the moral code. Psychopaths are capable of behaving morally just so they can function in society, but they do this simply by following learned scripts, not by avoiding behaviors that make them feel bad.
Because there’s a genetic link to psychopathy, it’s reasonable to assume that there are structural brain correlates for conscience, and fMRI studies of the brains of psychopaths seem to confirm this. So part of the answer to where conscience comes from is that it’s hard-wired into the brain. It makes perfect sense that a faculty like this would have evolved in a highly social species; the more individuals are able to self-enforce moral behavior, the less resources a society has to put into external enforcement of rules.
On a psychological level, most people have an aspirational self-image, the person that they’d like to be. We form that aspirational self-image from all kinds of social signals that we receive from our entourage as we grow up. It’s the kind of person who gets rewarded with love and acceptance and respect. We want those things, so we’d like to think that that’s who our best self is. The negative feelings, the pangs of conscience, come when we perceive that our behavior doesn’t match up with that ideal self. We’re disappointed in the reality of our self, because it doesn’t live up to the ideal.