In the autumn of 2003, I was in the press box at Stamford Bridge to watch Chelsea host Lazio. It was the Blues' first season with Roman Abramovich cutting the cheques, while the opponents were, by that stage, a spent force.

A few minutes into the game, Simone Inzaghi was tackled by John Terry. The referee waved play on, as the Italian striker continued to roll around and writhe in agony, real or imagined. Demetrio Albertini won it back and, seeing his teammate still on the ground, booted it out of play to allow the physios on. After a minute or so, Inzaghi was on his feet and Glen Johnson ambled over to take the throw-in. Angelo Peruzzi, the Lazio keeper, advanced to the corner of his penalty box with one hand in the air. Pretty unmistakeable. Lazio had put the ball out of play because Inzaghi was down; now they expected it back.

But Johnson ignored him and, instead, sent it back towards his centre-back William Gallas. Boos rose up from the Lazio end. Gallas knocked it back to Johnson. More booing, now sustained. The guy in front of me in the press box grew excited. Very excited. He picked up his phone and — obviously chatting to his sports desk — said "It's starting! It's starting! They've gone after Johnson and Gallas already... they're racially abusing him."

By this point, the ball had reached Frank Lampard. Still more booing. But my colleague, evidently, hadn't noticed. Several Lazio players were now throwing their hands in the air. Lampard touched it back to Claude Makélélé, who lofted it into touch, raising his arm and giving the thumbs up sign. What happened seemed pretty obvious to me. It was the old question of whether you give the ball back when the opposition puts it into touch because one of their players is injured. In Italy it was standard at the time. In England, less so. Makélélé, freshly arrived from Madrid, realised what was going on and did the right thing. He even got some polite applause.

All this, of course, was lost on my colleague. All he heard was the Lazio ultras hound and boo Johnson's every touch for the next twenty minutes or so. At half-time he picked up the phone again to call his boss. He was positively glowing with glee as he told tales of "vile racist chants." He wondered "when will they ever learn?" And he cracked a joke about how "laughable" the punishment Uefa would inflict in the unlikely event that they actually "did anything about it." Eight years on, I have a slightly better understanding of how the media (not just in England, but everywhere) works. And I'm not surprised that this guy seemed to have been dispatched to Stamford Bridge with the sole purpose of finding "a racist fan angle". Nor is it much of a shock that he was so happy when he found it.

We live in a (relatively) civilised world. Violence, anger and hatred are all frowned upon. Yet, for many, they're still very real feelings. And that's where stuff like this comes in. One thing you're allowed — or even encouraged — to hate is racism. An article about racist fans (unless it's the fans of our own club, of course) is actually comforting. It allows us to vent our hatred for the racists. And it makes us feel good because, well, we're not like them.

Fascists rank just behind racists in the list of people who it's OK, if not encouraged, to hate. (Paedopohiles, of course, would be top of the list, but few, if any, sets of fans identify as such.) Just as racists offer us lovely audio cues to their racism (booing, monkey chants, etc), fascists clothe themselves in right-wing paraphernalia and make easily identifiable salutes. All that is great and wonderful to those whom are determined to "fight racism and fascism" because they're so easy to spot and verbally crucify.

When the cues aren't quite as obvious — or you blatantly misread them, as my colleague did that night — well, that's OK. They're ultras or hooligans or the great unwashed/unenlightened. Mistakes will happen. (In some cases the misreading reaches grotesque proportions. I remember one newspaper having a go at "racist" Italian fans for calling Alvaro Recoba "El Chino" which means "the Chinaman", presumably because of his vaguely East Asian facial features. Never mind the fact that the term "El Chino" is actually Spanish and it has been his nickname since he was a kid. Or, in fact, that he used to hand out business cards emblazoned with the words "El Chino". And, if I may make a brief digression, it's worth noting that each culture has a very distinct sense of what is acceptable and what is not acceptable. That very same article about Recoba talked about his "oriental" features. Now, I went to a university that helped define political correctness at its peak in the early 1990s. "Oriental" was a term that was exclusively acceptable when describing rugs. Why? Because the word "oriental" carries with it a geographic bias. "Oriental" comes from the word "orient" which means "east". East of what? Why, east of Europe of course. Using that word places Europe at the centre of the world, to the detriment of other parts of the globe. Of course, in the UK, "oriental" as a term is fine and dandy with most people.)

But back to football. Whenever a racist incident occurs — and it usually involves a set of fans from southern or eastern Europe — Uefa investigates and doles out fines. Now, no matter how much the fine is, you can be assured the media will describe it as "laughable" or "derisory". €10,000? €50,000? €100,000? All "laughable" and "derisory". And there will be calls for Uefa (or, rather Uefa-and-Fifa, the mighty — and fictional — super-cabal controlling the world game in the minds of many who lump the two together at every opportunity) to "get tough" on racism.

Fine. Who would disagree with that? Nobody. But what is racism? That's when things get a bit tricky. Yes, the monkey chants and booing of black players are racist acts. But they're not racism. Doing something racist and being a "racist" are two wholly different things. And in our self-righteous glee at pillorying the racist chanters we often forget that. Those who engage in racist acts may be inveterate racists who are prejudiced and actively discriminate against those of a different skin colour. Or they may simply be people who do it to wind up opposing players or fans or to seek attention or to show how bad-ass they are, but, in real-life they are in no way prejudiced. Unless you've got mind-reading powers — real ones, not the Derren Brown variety — or can look into somebody's soul, the best you can do is identify racist behaviour: not racism.

And the reverse, of course, is also true. You might not utter a racist word in your life and not make a single, visible racist gesture, but you might still forbid your child to date a black person or refuse to promote a worthy member of a different race or simply hope that the black man on the bus doesn't take the one empty seat next to you. All of the above, by my definition at least, make you racist. And a far greater threat than the guy making monkey noises at Shaun Wright-Phillips or Mario Balotelli. (Of course, this is not to say that overt racist abuse should not be stamped out. It should be and in the most decisive way. Simply fining clubs evidently doesn't work very well. But there are other things you can do. Halting the game until the racist abuse stops has worked quite well in the Netherlands and Italy and elsewhere. What tends to happen is that the majority of the crowd simply get annoyed with the racist chanters because they've stopped the game, proving that few things are as effective as peer pressure in controlling behaviour. Another solution would be maintaining the system of fines but using the revenue to fund a system of "observers" at matches who would be responsible for identifying individuals and getting them banned.)

The real danger with the current situation is thinking that by banning overt racist abuse we trick ourselves into thinking that racism has been eradicated. Because nobody is throwing bananas at players in the Premier League, we reckon the job is done. And that's simply not true. Racial prejudice is alive and well. It's just driven underground. The fact of the matter is that you can take all the black Premier League managers, top-flight referees, high-ranking officials at clubs, the Football Association, the Premier League, the LMA and the PFA and fit them into a mini-van. Oh, and it's not just an English thing. Nearly half the population of Brazil is black or mixed-race, yet how many non-white Brazilian managers can you name? How many French ones, apart from Jean Tigana and Antoine Kombouaré? Dutch ones who aren't Ruud Gullit and Frank Rijkaard? Serie A has never had a black manager, domestic or imported. Neither has the Bundesliga. Spain has had two: Rijkaard and Francisco Maturana, while the Premier League is up to four (Gullit, Tigana, Chris Hughton and Paul Ince). The fact is that, unless they're players, agents or members of Fifa's executive committee, black folks — with a few exceptions — simply don't matter in football. But, of course, tackling an issue like that is far more difficult and uncomfortable than simply railing at a moron who makes monkey chants. It's a lot easier to write indignant features about racism in Russian or Serbian football than it is to effect a real change and ask ourselves why some people aren't getting a fair shot.

Right now, we're paying lip-service. We think that by elevating racist abuse and treating it more seriously than other kinds of abuse in football grounds we've done our part. Grame Le Saux played week in, week out in front of tens of thousands of fans chanting "Le Saux takes it up the arse!" and that was OK. I suspect that if Le Saux had actually been gay, rather than a slightly effeminate-looking Guardian reader, we might have treated it differently. But since he's straight, well, it's just banter, isn't it?

Or go watch Liverpool away. At some point, you'll hear, "You look in the dustbin for something to eat,/ You find a dead rat and you think it's a treat,/ In your Liverpool slum!" If you're lucky (or unlucky, depending on your point of view) you might get the third verse of that delightful ditty, which goes, "You piss in the sink and you shit in the bath,/You finger your gran and you think it's a laugh,/ In your Liverpool slum!"

The above is supposedly banter, because it's supposedly funny, because Scousers are (mostly) white and English, just like those doing the singing. But stray into Emmanuel Adebayor's "elephant song" and it's racist abuse. Why? Because of the line "his dad washes elephants, his mum is a whore!" Note that it's not the part about Adebayor's mother that's not OK. It's the business with the washing elephants, since elephants come from Africa and Adebayor is black (and, presumably, so is his dad.) Elephants, of course, also live in Asia, but, I guess, since there are so few Asian footballers in the Premier League, nobody has made the connection. If all the energy and bile devoted to decrying the Adebayor song had been dedicated to doing something about the woeful under-representation of black people in positions of power in football's food chain, might we be marginally better off? I'd like to think so.

This hierarchy of abuse of course extends to players as well. Think back to the 2006 World Cup final. A couple of charlatan lip-readers (God forbid anyone of us ever finds him or herself in court with one of these expert buffons on the witness stand) decide Marco Materazzi has racially abused Zinédine Zidane and, suddenly, the fact that he delivered a cheap shot head butt is OK. He's fighting racism too. (Zidane, to his eternal credit, had the decency to reveal that Materazzi's words were not racist in any way. He could have taken the easy way out and remained a hero to the sort of itchy-fingered anti-racism types who immediately crucified Materazzi. And it probably would have salvaged some of his reputation. Instead he — eventually — took responsiblity for his actions.) Of course, when it turned out that Materazzi had "merely" offended Zidane's sister, well, then it was Zizou who was a bit unhinged. After all, calling a woman a "whore" is nowhere near as serious as labelling someone a "terrorist" or a "son of Harkis".

Or how about the infamous incident between Siniša Mihajlović and Patrick Vieira more than a decade ago? Because they're both grown-ups who have given their version of events, we now know the truth. They clashed on the pitch. Vieira called Mihajlović a "shitty gypsy", Mihajlović called him a "shitty black". (For some reason much of the British media persisted with the — entirely made up — notion that Mihajlović had called him a "black monkey". It's irrelevant of course in the sense that both are clear examples of racist abuse but it does make you wonder about the accuracy of the stuff you read.) Mihajlović admitted his offence and apologised and got a two-match ban. Vieira was depicted as the victim. And that makes sense: we Europeans have a racist history of black people being discriminated against and enslaved. (Though, to be fair, I'm not sure it was the ancestors of Mihajlović — who is half-Croatian, half-Serbian — who were profiting from the slave trade a few centuries ago.) But you can't help but wonder: if that had been Vieira's Aryan-looking teammate Emanuel Petit who had called Mihajlović a "gypsy" would the outcry have been similar? Or was this story too convenient to pass up: a thuggish Serbian with links to Arkan hurling racist abuse at a young black man? Mihajlović remains a figure of hate to this day, the "gypsy" comment which Vieira, to his credit, owned up to almost immediately, was quickly forgotten. After all, we treat abuse on a sliding scale.

At some point we probably need to grow up and agree what we will and will not allow in our football grounds. Right now, anything goes, except for racist, anti-Semitic and — but seemingly only if Rangers or Celtic are playing — sectarian abuse. And, at some point, we need to ask ourselves why we ban certain speech in football grounds. Is it to make some sort of social statement? Is it simply a commercial decision, to make our stadiums more inviting and pleasant? Is it to make ourselves feel better?

Increasingly, much of the anti-racist stuff in the mainstream media is motivated primarily by the latter two. Nothing wrong with that, of course, unless it lulls us into a false sense of achievement, a sort of "pat-on-the-back" self-congratulatory attitude which leads us to ignore the reality that racial prejudice — if not outright racism — is real and all around us.

The thing to remember is that football is not going to fix that. Football can modify behaviour, but it can't modify thought processes. It takes society and evolution do that. And that will take time.

This article appeared on Episode Eighty Six of the Blizzard Podcast. You can see which other articles we have featured by searching the Podcast tag, but we'd really like you to subscribe which you can do through iTunes, Soundcloud, our RSS feed or wherever you usually get your podcasts of choice.