It’s not a very expensive hobby, but do I find it saves over buying commercial beer? Nah. I’ve made weak ales for under 50 cents a bottle, but that is not counting the bottle itself (these are reusable), cleaning, or my own time. But a good homebrew is so much better than a commercial beer.
I suggest avoiding the kit @wildpotato mentioned. There is nothing wrong with it per se, but it’s not a lot more work to make 5 gallons than it is to make one gallon. Those kits are aimed at people with tiny NYC apartments! Also, in my anecdotal experience larger quantities turn out a little better. Also, avoid Mr. Beer. Get something like this to start out. You’ll quickly want to grow your equipment out as your skills expand, but this stuff stays useful.
One thing you’ll want quickly is a large stockpot. I suggest at least 8 gallons for 5 gallon batches, and maybe 10 gallons for slightly larger batches – I do 6.5 gal.
I don’t use bottles so much anymore, but I used to get my bottles from the local beer distributor, which also handles glass bottle recycling. It gives me my bottles for 5ยข, the cost of bottle redemption. I then soak them overnight in a bath of sodium percarbonate (B-Brite or OxyClean), which cleans them and takes the glue and paper labels off. Also, I look for larger bottles like Tecate quart bottles.
Obviously, you can spend thousands on this hobby. I have an outdoor propane cooker (this model – highly recommended). Kegging equipment isn’t exactly cheap. And there’s refrigeration, temperature controllers, etc..