It’s not going to be a problem for the simple reason that these days almost no one else has to handle your credit card. About the only time I don’t swipe my card myself (or use it over the phone or computer) is when I use my AmEx card at a restaurant. That’s pretty much the only time that someone else rings up a charge for me. So, since no one else normally even sees my card, its appearance can hardly matter to them.
What I would be concerned about, however, is the durability of the magnetic strips if the cards continually rub against each other or are jostled in a pocket or purse with keys, pens and other potentially damaging metal objects.
Edit: I would be sure to double-check any place that I planned to punch a hole in a card, though. You’ll probably want the hole in the same place on every card, so they appear neat on the key ring or holder. So check to make sure that you won’t damage the raised lettering or numbers on any card, the magnetic strip on any card, the “security number” on any card, or even the contact telephone numbers or signature strip on the back of any card.