Are you ovulating? Do you know? Because… I had an egg that went a bit awol once. It left the ovaries and didn’t go down the tubes. It landed outside my reproductive system and my body, of course, treated it like a foreign body and created a bit of an ‘immune system’ party around it. The pain went away after a while, but it sure did hurt. But it was like you mention, if he hit in one spot, or was too deep, there was pain.
Cysts on your ovaries is a serious business, as @JustJessica said. It runs in my family. I’m glad I didn’t get it, but two sisters and two nieces are afflicted. I know the stories. Yours doesn’t sound like it.
If this persists, see your doctor, in the meantime, explain things to him, be honest and comfortable during sex. It shouldn’t hurt.