Yes, "don't leave me" is right - but the "official" English song translation is "If you go away.." Probably because this phrase has the same note of desperation as "ne me quitte pas," while still leaving some room for the follow-up lyrics (and keeping the same meter as the original for the song). Emiliana Torrini (Italian/Icelandic singer) does a pretty great, breathless version with the English lyrics. In on her "Rarities" album.
But please don't listen to this song too much in French or English! I've made that mistake before, and it sure puts you in a blue mood. You've been warned...