Is it discriminatory if you notice patterns?

I grew up in a poor neighbourhood and I learned to hate being bullied by certain groups? Am I racist? I don’t think that I need to identify the group that I am scared of. I got tall and most of the bulliying ended. Now the group just panhandles me with new tricks.