Depends what you call “the modern calender.” The current iteration is the Gregorian Calender, and it basically dates to 1582. It fixed some bugs in the Julian calender, which was in use since Julius Ceasar (40s BCE, if I recall, either way decades before Christ). That, in turn, is based on even older ones.
It also depends on who you include in “we”. The year cited by @bolwerk is when the Gregorian calendar was first put in use, and was adopted by Catholic countries. But the British Empire, including the thirteen colonies, did not adopt it until 1752. Alaska (always behind the times) did not adopt it until it became a US possession in 1867.