A bigger issue to me would be how parties in power (most recently the Republicans over the last 10 years) zone the districts to favor their party.
For example you’ve got a big city like Columbus (my city). Its surrounded by large rural areas for quite a ways in every direction. Not surprisingly the city is largely liberal/democratic/left and the rural areas are largely conservative/republican/right. Combined the areas have enough population for two zones and hence two seats in congress. Now rather than split this into the city and the rural areas, essentially guaranteeing each party one seat and guaranteeing that the majority of people in those respective areas have their choice for candidate in office…. The Republicans some years ago zoned it so that Columbus is sliced right down the middle, and each half is combined with half of the rural area… plus just a bit more rural area.
The result is 2 seats that are almost always Republican, even though the vast majority (70+%) of the cities voters, whom would themselves have enough populace for one congressional house seat, have no voice in congress.