They are most likely young centipedes, because I’ve had them in my potted plants before. They most likely were in the pot (when you bought it at Wal-Mart) before you re-potted your plant, not from the bag of soil you bought. Almost all houseplants that you buy at garden centres or department stores come from tree farms in warmer areas of the country. There, insects often are indigenous to the soil, and they were already in the ground when the plants were potted up for selling, or they get into the pot easily, since the pots are outside under cover. Centipedes (and millipedes) will not harm your plant. As a matter of fact, they prey on other bugs in the soil, and eat decomposing organic matter. However, they are yucky, and I don’t like it when I see them inside the house. What I would do is remove the dead insects on the surface, then water your plant with a very soapy solution (dishwashing liquid or an insecticidal soap that you buy at the garden centre or Wal-Mart is the best). Completely soak the soil with the soapy water. Do it outside if possible, so that you don’t get the solution on your floor. Keep watering the soil with the solution until it is drenched. The solution should be really soapy and bubbly. It will effectively kill any centipedes and insects in the soil, and leave your plants unharmed. I’ve tried it on my hibiscus plant, and no more centipedes!