What makes anything wrong? As soon as you have an idea about the way things ought to be, then Right and Wrong appear: Right is that which tends toward how things ought to be, while Wrong tends away from how things ought to be.
If you think, “I hate being in constant fear of my life. I see that most other people feel the same way. The world ought to be a place where we mutually agree to let each other live, so we can be relieved of some of that fear”, then killing starts to look quite wrong.
That leaves the door open for exceptions. Suppose a gunman starts picking off passersby. Because his existence now poses an imminent threat to the lives of many others, killing him starts to look right.
Much depends on how the “ought” idea is formulated. If you think “The world ought to be a place where Aryan people can live untroubled by non-Aryans” then, as we now know, killing can start to look right. If you think, “The world ought to be a place where I can do anything I damned well please”, then killing may also start to look right.
The “Golden Rule” is just a policy on how to formulate good “ought” ideas.