You can remove Internet Explorer from your Start menu by right clicking it’s icon and choosing ‘delete’ or ‘unpin from Start menu’ depending on the instance in which it appears. The same is true of the Desktop and Quick Launch icons.
If you ever need to launch it in the future, choose ‘Start’ and then ‘Run’. Type “iexplore.exe” (without the quotes) and then click okay.
The caveat with using a non-IE browser is that Microsoft have so tightly integrated it with the system throughout XP, and it still cannot be properly uninstalled even in Vista. You will notice that the software update manager will always launch in IE throughout XP (isn’t a web browser a silly place to manage system updates?) and that some links from Microsoft applications like Live Messenger will always open IE even if it is not your default browser.
This said, I would wholeheartedly recommend Firefox. Do not look back. It is much easier to use, they don’t bury useful menu items under a plethora of strange icons, and I like the way it handles tabs and bookmarks much better. Your history, for example, is much easier to browse in Firefox than in IE.
To switch Firefox to your default browser, launch it for the first time and select to do so on the prompt that will appear stating that it is ‘not your default browser’.
Safari is worth keeping an eye on, but I wouldn’t bother using it on the PC yet. I’m really a Mac user, so I do use it on that platform, but I don’t think it’s ready to be a PC app quite yet.
Good luck, hope this helps.