People have been asking this for a long time. On one hand, you have Plato’s perfect forms, and on the other, Aristotle’s ideals.
Plato said that there was a realm of perfect things, and the things we see here are mere shadows of that perfection. For instance, we can’t draw a perfect line or a perfect circle, we can only approximate the perfect shape that exists in that other realm.
Aristotle didn’t believe in any such perfect forms. He said they didn’t exist, that nothing was perfect. However, in our heads, we know what a perfect circle or line ought to be – we can conceive of it, even if it doesn’t exist. The idea of the thing exists in our minds.
Now, if you believe in something like Plato’s perfect forms (and one could say that the idea of Heaven borrows heavily from this concept), we might say that we “discover” the mathematical object – or, at least, the shadow of it that manifests in this world. And if you believe that mathematical concepts mostly exist in the mind of the thinker (because, after all, if there’s no mind to think a theorem, does it exist?), you might say that we’ve invented the object to explain a concept.
If you figure out a way to beat Plato or Aristotle at this game, let me know. ;)