It depends on who the person is and why they are ‘going away from me’. Perhaps it’s a relief to see them gone if they are a friend and the relationship has become toxic. If I’ve also reached the point where I know a relationship has run its course, I might feel glad to see the end of it.
If it’s a family member or a loved one, even if the relationship is toxic, I’ll undoubtedly have regrets and if I’m not ready to let the person go, the regrets could be quite extreme and I’ll hurt a lot. I have broken off contact with family members and I’ve had family members break off contact with me. It hurts. Even if I know it’s good for me, it still hurts and I wish things were different.