If you have a group of people living near you who all wear the same clothing, same hats, speak using the same slang, listen to the same style of music, drive cars modified in the same way, and who invariably attack you, steal from you or vandalise your property, and you still keep trying to approach them and be friendly, you will be regarded as an idiot. It is a gang, a closed group you will be told, they do not want to accept you, they will always be a risk to you. Avoid them!
If you add that these people are all members of a particular race, you be called a racist, told not feel prejudice, to be welcoming and friendly toward them, no matter what response you receive.
It can be hard to walk both lines. It is also completely unrealistic as disliking some negative constants in a race is NOT racism, it is normal. Other members of the same race also dislike it.
I have no racial prejudices, but I definitely do have behavioural prejudices, and if that behaviour has a race bias – tough. It isn’t their race that earned the dislike, it is their behaviour.