Here’s a somewhat technical explanation.
RSS is simply put a machine readable text file that you can download to your computer. The RSS file will contain information about the latest posts on a blog or maybe the latest comments in a forum. If you let a newsfeed reader download it the reader will compare it to earlier versions of the file and let you know if there is something new you could look at.
You should know that there are some different versions of RSS and even a newer format called atom. They pretty much all do the same thing as far as the normal user is concerned. If a site asks you to choose a format just go with RSS 2.0 or atom. You can read more about the difference in this thread if you are interested.
These feed formats can also have embedded content like audio or video, and that is what podcasting is if you have ever come across that.
About readers. There are many that like using web based services for this, like Google reader that @justn mentioned. However, if you use a common browser you could use the built in reader. I really like the ones in Internet explorer 7, Flock and Safari. Some other suggestions here.
I hope that made sense..