Distrust of people different than yourself is a basic human instinct, whether it's different cultures, clans, tribes, races, or class. Denying this exists is a ridiculous concept.
Now if you want to try to rise above all that, more power to you. It is difficult to unlearn instinct, but it can be done.
The instinct is there for a reason. It's a survival mechanism. Learning to override it is important for trade and diplomacy. In our modern society, these things still exist, of course. Racism is the one that gets all the attention, but there is classicism, culturalism, and even tribalism deeply ingrained into even the most liberal thinking modern man. Just look at the different uniforms we have in our society. Coats and ties indicate business culture and/or success. Jeans and work boots indicate working class, pants down around the knees indicate outlaws. To deny profiling using this method would be stupid and less than constructive.
Humans are sentient beings. We are capable of lying, misinforming, and general deception. Other than natural camouflage, no other animal is capable of any of these traits. So keep in mind that just because someone is wearing a suit, it doesn't make them a businessman. They could be a scam artist. Many of the young people today imitating the dress and demeanor of outlaws are not outlaws at all, but are wearing the disguise as a means of defensive camouflage, or to simply bolster up some lack of self esteem.
So the first thing we need to do is realize that prejudice is not the same thing as racism. Racism is a form of prejudice, but I can prejudge someone based on dress, accent, immediate actions, type of car they drive, etc.
So if I see a white guy wearing a suit walking towards me and I make no defensive posture, no matter how subtle, I have prejudged. If I see a black youth wearing the thug uniform and I switch sides of the street and keep my eyes on him, I have prejudged. But is it because he's black, or because he's dressed in a tribal/cultural uniform that I've learned to distrust and/or don't understand? Or maybe a bit of both?
This is where racism (and I mean the modern use of the term, not the literal definition) falls down. We concentrate so much on the color of the skin, that nothing else is important. I moved to the other side of the street because I was prejudiced, but was it because I was racist? Maybe if a black man in a suit came down the street, it would barely register any more than the white guy in the suit?
This is where our current hysteria over racism falls down. All we see that is important is skin color. Actions, uniform, attitude, circumstances... none of these things are important. It's all about skin color. But it reality, we prejudge in every interaction we have all day long. Prejudice is a simple survival instinct.
Az