Er, almost, but having a lease generally means that the rental contract is for a period longer than one month.
In general, either you are “renting month to month”, meaning the rental contract says you might choose to leave (or that the owner might raise the rent) at any time, with no penalty or obligation, or alternatively you “have an X month (or year) lease”, meaning that you agree to rent it for some number (X) of months, and the party renting it to you agrees to let you rent it for a fixed amount. If you have a lease but want to stop renting sooner, the contract may specify a penalty, for instance that you still have to pay until someone else rents the apartment. Also it is common in many places for a rental contract to specify a lease period but after that period (often one year), the contract becomes a month-to-month rental unless the renter elects to renew the lease for another year.
In both cases, there is a rental contract, but “lease” typically implies a period of fixed conditions longer than a month.