Glass is definitely inert, but it can crack. Stainless steel is also inert (unless it is some modern crap stainless steel with insufficient chromium in which case it will rust way before it is supposed to) and not so brittle.
Based on personal experience I will say that if you store water in plastic for a significant period of time, it acquires a “plasticky” taste. Whether or not carcinogenic toxins are leaching out into the water from the plastic, I can’t be sure, but it sure tastes like it is.
If you are using plastic, make sure you keep it out of sunlight and heat which both can degrade plastic.
I hate everything about bottled water, but it sounds like you’d be better off buying a case of gallons of water and just stashing that. Then you don’t have to worry about bacteria because the bottles will be properly sealed.
And can I ask why 10 days?