The thing to look at is the platform. For hosting web apps, Windows/IIS/MSSQL/.NET is a common platform. The other heavyweight, probably the most popular, is Linux/Apache/MySQL/PHP (LAMP is the popular acronym, but the P could stand for PERL too). And there are variations on that, like FAMP (FreeBSD/Apache, etc. – my preference) or WAMP (Windows, Apache, etc.). Maybe MAMP too, though I can’t imagine anyone doing seriously server-side development on a Mac. Probably the most notable thing about the out-of-the-box Windows option (besides that it really is turnkey and out-of-the-box) is that it is proprietary. I agree with @HungryGuy that proprietary software is something to avoid as much as you can, and I manage to avoid it completely for my clients.
IMHO, some kind of PHP/*N*X solution is to be preferred. Not to say there aren’t drawbacks – if you have to maintain the systems yourself, maybe you would need to actually learn a UNIX-like operation system, and it can be a little daunting for a newcomer. At a minimum, a background in UNIX-like file system organization is a big help even if you hire someone else to host your site. Configuration is often manual using text files. To me, that’s easier, but I’m used to it.
I’m personally not very comfortable with Windows’ vender lock-in, but if your skillset fits it better, maybe you’ll prefer it. But if you’re starting from scratch and have no idea what you’re doing either way, I would just go with LAMP. It’s more common and has fewer legal encumbrances – which may mean nothing to you, until it does.