For the right to not listed or be subjected, I’m thinking about harassment, though there’s arguably a sliding scale. There are rights and laws for that. What if I have a crazy ex who would like her freedom of speech to include phoning up me and anyone around me and us having to listen to insane rants, pranks, screaming, etc. Well I can get a restraining order. How about if someone wants to follow others around and yell at them – harassment, or disturbing the peace.
In the work example, I’d say everyone still has their rights to speak and to not listen. It’s just that there are natural consequences of using, abusing, or not using them, as with anything else, and a two-way work contract involved which would tend to be impacted with less provocation, before rights came into question.