If I’m the host—as I am, rarely—I take it as a near-sacred obligation to include everyone. Even when I’m not the host, I try to be aware of who’s not involved or included in the conversation, and include them. (More often than not, it happens to be me, and then I just try to smile and appear open to conversation.) I know how uncomfortable it can feel to be the odd man out, and I try to make sure that others don’t feel that way—unless they seem to want to.
I once read an article on conversation that said a good conversationalist should be able to speak about anything at all, from “world peace” to “helicopter maintenance”. And I’ve actually started some conversations along the lines of “Would you like to discuss world peace, helicopter maintenance, or some other topic of your choosing?” Usually the person is so hooked by that one topic that they weren’t expecting that they ask me if I’m in the helicopter business, and then we start a conversation about who-knows-what (starting with “no, I’m not in that business; what do you do?” etc.).
But here in Connecticut there are several companies who build, test and make control systems for helicopters, and once I tried that conversational gambit on someone I didn’t know… and got a dissertation on helicopter maintenance. So I tend to raise other topics now as potential starters.