These things are extensions, usually written by independent people (sometimes by the company themselves), that extend the use of the application. For Office, this includes new functions. I have more experience with browser plugins so I’ll talk about them. Here is the Firefox official addons page. These are written by third parties. Browse through them (here are the most popular ones) and you will see that each offers one new functionality that the browser doesn’t come with. For example, this one will let you change your browser’s color. This one hides a lot of ads from the Internet (but very occasionally causes loading or other problems).
There is a special addon called Greasemonkey. This is a plugin in a plugin if you will, but the only thing this controls is the visual layout and design of webpages. You can go to this website and find third party scripts that do additional modification of content of webpages. With Greasemonkey, I could create a script that changed the background color of this page, or removed the Amazon ads, or even create a script that would highlight comments I the original author post (good for skimming maybe). Or, extend this to highlighting comments (or hiding comments) written by certain users you have in a list. This is how one could create a “hide your hated users” function on this site, without Ben or Andrew being involved. Or a “highlight your favourite users” page. Etc.
In general though, the term plugin, addon, etc signify that it is a third party component with generally one function that adds or modifies functionality in your program.