If it’s cutting out momentarily, when you switch from one input to the other, it’s because it has to sync up with the signal from the device. This might not even be the fault of your audio system; it could be the TV itself. If you have problems with the signal dropping out intermittently long after you’ve switched the inputs, you probably have too much analog attenuation in the signal path. The first thing I’d try there is a better-quality cable.
Some sound systems have HDMI inputs of their own, and take over the task of switching the video from one source to the next, keeping the audio internal. That’s the way I would go if I were to start building a system from scratch.