You mentioned that Cable is included in your rent. Have you thought about asking if they will take that off and go lower on the rent? Its not needed, and there is no telling how much that is effecting your weekly/monthly rent bill.
Power is an area that’s easy to save on, but easy to forget about. Unplug TVs, computers, and other electrical devices which don’t need to be on at night. If its plugged in, even off, it still pulls a little bit of power. If there is nothing on a charger, unplug it! That charger is always pulling power. Change out your light bulbs to the compact florescent bulbs. They are more expensive starting cost, but last for years,and those that don’t generally have a warranty of 5 years if you keep your receipt. If you have windows, open the curtains instead of turning on the lamps and lights.
You said you can’t adjust the thermostat cause of a newborn, but 2 to 5 degrees makes a HUGE difference and isn’t noticeable off hand. Blankets help during the colder times to.
On the food side of things, do NOT buy water! 90% of bottled water or sold drinking water is straight from tap anyway, and, with a reverse osmosis filter put on your sink, you can actually make your tap water a lot better for you. The difference in cost? Pennies on the gallon for tap, compared to roughly a dollar a gallon store bought. Do not buy Tea already made, as making it yourself will save you a ton as well. Tea bags are cheap, sugar is as well. For the amount of money you spend making tea, you can make 3 to 7 gallons for what you pay for 1 from a store.
The thing about saving money is if you are willing to sacrifice convenience for money. Some of it sounds difficult, but trust me, people lived rather healthy without power, fast food, and cable long before we were here, and if we just monitor what we do and how we do it, we can live a lot cheaper than what we realize.