Hummingbirds are drawn to flowering plants for their appearance and stick around if they produce nectar. Your best bet is to check with a local nursery or bird-watcher group to find out the ideal types that will flourish in your region.
Hummingbird feeders are a nice addition that can allure them for a closer view. The key is change the sugar water regularly. The recipe is one part sugar to four parts water. The feeder also needs to be thoroughly cleaned at least once a month, as they tend to grow mildew.
On a side note, they are entertaining to watch. They are territorial, so when one stakes claim to the backyard in summer months, they often go into attack mode when another enters the garden. They are also quite curious. If you sit outside often, you may be able to get a really close look at one as they grow accustomed to you. There have even been several success stories in having hummingbirds feed out of your hand.
They also remember a source of food, so during the migration process, they often return to the same spot. Choosing nectar-producing flowers that bloom or last through early fall will increase the number of hummingbirds to your garden and their return.