For all the intrigue, surprise and enthralling football at the World Cup, the domination by Europe and South America, the sport’s old order, remained the same. The two continents account for 15% of the world’s population, yet retain an iron grip over the men’s World Cup. In Russia, just two of the countries to make the last 16 were from beyond these two continents. And Japan and Mexico both did as upstarts are meant to when they deign to reach the World Cup knockout stages: they were immediately knocked out.

Over 21 editions of the World Cup, only two nations from beyond Europe and South America have ever made the semi-finals. Even these two benefited from great fortune: the United States, in 1930, from the draw; South Korea, in 2002, from home advantage and some extraordinary refereeing decisions – including, in the second round victory over Italy, by Byron Moreno, who was subsequently convicted for smuggling heroin.

South Korea’s stirring performance 16 years ago, in a tournament in which Senegal defeated France and Sweden en route to the quarter-finals, was meant to usher in an age in which World Cup success was opened up to the world. Instead, recent tournaments have seen Europe more dominant than ever. In the last four World Cups, 13 of the 16 teams to make the semi-finals are European – the rest, naturally, are from South America. More extraordinarily, nine different European countries have reached the semi-finals in this time. Europe’s dominance is not underpinned by a few elite teams so much as an entire structure which has been endlessly tweaked and refined to drive the continent’s success.

Football is a universal game. But, as Russia 2018 reaffirmed, its hierarchies are not.

When the World Cup was created in 1930, no countries from Africa or Asia were invited to take part; 11 of the 13 participants were from Europe and South America, alongside Mexico and the USA. This set the template for how the World Cup evolved: more as a Euro-South America Cup with a couple of token representatives from elsewhere than a genuine global format. In 1962, 15 of the 16 countries to play in the World Cup were from Europe or South America.

Four years later, African nations boycotted the Cup in protest at a qualification system that awarded Africa, Asia and Oceania combined only a solitary place. Stanley Rous, Fifa president for 14 years until 1974, argued that the standard of play in Africa and Asia simply wasn’t great enough for both continents to merit a guaranteed place. From 1934 to 1970, no African nations took part at all; Europe even provided nearly all the referees. Fifa’s disregard for Africa was such that, before 1982, foreign clubs did not have to release African players for the World Cup, according to the historian David Goldblatt.

The impact of these years of neglect can still be felt. It allowed Europe and South America to develop a first-mover advantage over the rest of the globe in footballing expertise and infrastructure. For the best European and South American countries, the institutional experience of reaching the World Cup could be augmented every tournament, allowing lessons to be built upon. For countries from elsewhere, places were so sparse that there was little know-how of what to do during World Cups, or experience that could be passed on from one tournament to the next. And it was not until the new century, and the 17th World Cup, that a nation from Africa or Asia was allowed to host the competition.

The legacy of the World Cup’s development can still be felt in the allocation of places to different continents: this year, 19 of the 32 teams were from Europe or South America. But Africa and Asia still had 10 representatives in Russia; had places been allocated based upon the Fifa world rankings before the tournament, Africa would have had four and Asia none.

In different ways, the qualification process hinders teams from beyond Europe and South America. Consider Mexico who, despite a stirring victory over Germany, ended this World Cup just as they had the previous six: lamenting the curse of “quinto partido” – the fifth game, which they are always denied. Considering their population of 127 million, wealth and huge interest in football – Hirving Lozano’s goal against Germany triggered celebrations so wild that they registered on earthquake detectors – Mexico can be considered among the most underperforming football nations.

One reason for Mexico’s failures to translate their multiple global titles in underage football into the senior game is their lack of exposure to high-quality teams in qualifying. In their matches in Concacaf – the region for North and Central America and the Caribbean – Mexico are almost invariably favourites, used to having a high share of possession and dominating opponents. Yet waltzing through qualification comes at a cost. It is shoddy preparation for the challenges of the World Cup – above all, in the knockout stages.

African teams have a different problem. With five places for the entire continent, there is huge volatility in the qualifiers from one World Cup and the next: only one of the African teams who made it to the 2014 competition returned in 2018. The churn of sides encourages short-termism, at the expense of longer-term planning, and means that countries frequently go to the World Cup with no experience of the competition in their squads.

Still for all these teams the central issue is the same. They go four years between facing football’s international elite in competitive matches, and are then thrust into the most high octane games in the sport. Europe’s countries are continuously tested by a rigorous qualification process that ensures highly pressurised, competitive matches. And so the passage between qualification and the World Cup itself is smoothed.

In economics the middle-income trap explains why the growth rate of developing countries slows down when it reaches a certain point. Essentially, middle-income countries can copy technologies that have worked in more developed countries, up to a point, but cannot develop the new technologies and innovation they need to catch up with the most economically advanced nations. The World Bank has shown that, of the 101 countries that could be classified as middle-income in 1960, just 13 were high income by 2008. The closer poorer countries get to richer countries, the more their economy slows.

So it is, too, in international football. Countries from Africa and Asia who reach the World Cup have achieved a level of fitness, professionalism and skill which renders a repeat of past ignominies – like Zaire’s 9-0 loss to Yugoslavia in the 1974 World Cup, or Saudi Arabia’s 8-0 defeat to Germany in 2002 – almost inconceivable. But as football’s emerging nations have improved, so it has become harder to make further progress.

This is football’s equivalent of the middle-income trap, as the economists Stefan Szymanski and Melanie Krause demonstrate in a new paper. Analysing over 25,000 international football games, from 1950 to the 2014 World Cup, the authors find the middle-income trap at work in international football.

Until 2002, emerging football nations were like emerging economies – they were still behind the elite, but they were advancing at a faster rate. Szymanski and Krause found that the success rate of African teams against those from Europe and South America (treating a draw as half a win) was 23.3 per cent from 1974-8; by the 1998-2002 cycle, it had risen to 40.9 per cent, meaning they won four games against European and South American sides for every six they lost. During the 1998-2002 cycle, countries from Asia and Concacaf also performed similarly strongly against teams from Europe and South America. These three regions were not yet at parity with Europe and South America – but they were exhibiting continuous improvement.

So the successes of Senegal and South Korea in 2002 were in keeping with their continents' improvements on the world stage. The first World Cup of the new millennium seemed to represent a staging post for Africa and Asia to improve further – and, ultimately, to launch serious assaults on the trophy. Yet now 2002 looks more like a high-water mark: false hope for the emerging football world before they entered into a phase of relative decline.

Ever since the 2002 World Cup ended, the rest of the world has gone into a slump against Europe and South America. Asia’s success rate decreased from 41.3 per cent in 1998-2002 to 30.8 per cent in the 2010-14 cycle. Africa’s slipped from 40.9 per cent to 33.1 per cent. Concacaf’s fell from 48.6 per cent to 33.4 per cent. Countries from these three regions have improved since 2002 – it’s just that those from Europe and South America have made more substantive gains, so the gap between football’s old order and upstarts has actually increased.

The 2018 World Cup showcased the most significant trend in international football since 2002: the rise of smaller European nations. Three European countries of under 12 million inhabitants – Belgium, Croatia and Sweden – reached the quarter-finals.

These triumphs embodied that small European nations have been the biggest risers in international football in recent decades. In their analysis, Szymanski and Krause analysed the performances of a sample of 76 countries from around the world - each with a population of over one million – since 1974. Comparing teams’ performance in the period 1975-86 and 2003-14, the authors found that European teams – who already had the highest average ranking in the first period – improved their performances the most. South American countries, the second highest-ranked in the first period, also improved. Meanwhile the ranking of Asian teams remained static, while teams from Africa and Concacaf declined. Essentially, the better a continent was before, the more it improved.

This suggests what has happened is a form of consolidation, with the best countries becoming even better. Actually, the authors identify a different trend: Europe’s improvement was not driven by its best teams but those who used to be among its worst. Comparing the rankings of nations from each continent in the bottom half of the rankings from 1975-86 with where they were in the next sample, from 2003-14, Szymanski and Krause showed that European countries previously in the bottom half of the sample improved dramatically. These teams increased their average ranking from 52nd to 37th from 1975-86 to 2003-14, while the African nations previously in the bottom half slipped from 50th to 52nd. “The biggest beneficiaries of worldwide convergence have been second-tier national teams from Europe and South America,” Krause and Szymanski write. Football nations ranked outside the elite have better prospects of improving rapidly if they are in Europe than anywhere else.

The most obvious advantage is of competition. In Europe, smaller nations are concentrated in the world’s dominant football federation. So lower-ranked European countries have regular opportunities to play and learn from better teams that are the envy of their equivalents in Africa and Asia. The opening up of the European Championship – from eight teams to 16 in 1996 and then from 16 to 24 in 2016 – has also afforded lower-ranked European sides more chance of reaching tournaments, thereby lessening the sense of the unknown that accompanies reaching the World Cup. Were it possible to take two identical countries and put one in the European confederation and one in the African one, the work of Krause and Szymanski shows that the side in Europe should be expected to rise at a faster rate.

But in football geography matters for far more than which confederation it places a national team in. European nations have access to the sport’s best football knowledge network – in which ideas, technology and best practice can diffuse seamlessly across borders. Asia and African teams, constantly hiring veteran European coaches, are like generals fighting the previous war.

The better-connected a side is in Europe, the better they tend to perform. The rise of Spanish and Portuguese football came a generation after their ascension to the European Union, in 1986, as Simon Kuper has noted. Being in the hub of Europe makes it easier to bring in the best coaches – and for the best young players to learn from the best footballing minds, even if that necessitates moving abroad.

European players in countries within the European Economic Area have a particular advantage: they are allowed to move to other European nations aged 16 and 17 while, for players outside the EEA, Fifa prohibits all overseas transfers of U-18s. This rule has been especially useful to the smallest countries. Iceland’s squad is brimming with players who moved abroad shortly after turning 16: Gylfi Sigurðsson signed for Reading less than a month after his 16th birthday. That players from other continents are unable to move overseas aged 16 or 17 – understandably given the history of young footballers being exploited – is a significant handicap, believes Paul Darby, the author of Africa, Football and Fifa. “These opportunities are not available to young African players. Given that the game has become much more tactically and intellectually demanding in recent years, the loss of two years of high-quality football development may be impacting on African performance at the World Cup.”

Even after players turn 18, there is still not a genuine free market in football talent. Most European leagues have strict eligibility rules for overseas players – in the Premier League, players must be regulars for international sides in the top 60 or command a transfer fee and wage above the median. This is a roadblock to many foreign players signing – except those from within the EEA, who can move without fulfilling any criteria. And so while all players from the smallest European nations are free to sign for any club in the continent, those from Africa and Asia are not.

Football’s hyper-professionalism has been another boon to European countries. While sport fetishises the notion that children are more likely to make it to the top if they are ‘hungry’, living in richer countries confers great advantages – better nutrition and more space to play – to young footballers. Europe’s youth systems have been meticulously honed to maximise players’ chances of making it to the top: consider the small-sided underage games children play at Clairefontaine, La Masia or in Belgium, where they are credited with transforming the side after a poor performance when they co-hosted Euro 2000. The revolutions in sports science, technology, analytics and nutrition have created new areas for Europe to gain an advantage, explaining why it has left the rest of the world behind in the past 15 years; Mario Götze’s World Cup-winning goal in 2014 was famously akin to a move on Footbonaut, the training cage developed in Berlin which fires balls at a player from all angles.

“Europe has actually leapt ahead, it hasn't stood still. The preconditions for success in 21st century football are slightly different from the 20th century because money, technology and organisation is being brought to bear in a way that it never was before and Africa is losing out. It can't keep up,” believes Goldblatt, author of The Ball is Round: A Global History of Football.

“I’ve been coaching for 37 years and I can tell the gap between Europe and the rest is only growing. Eight years ago it was bad, four years ago worse and now it’s even worse,” Carlos Queiroz said after Iran’s defeat to Spain in the World Cup. “In Asia, the development process is struggling at both national and youth level and by that I mean absolutely everything. It means coach education, youth development, infrastructure, facilities – everything – Europe is taking off and the others are being left behind.”

European nations are the richest anyway. And the continent’s football federations are buttressed by being in Uefa: the wealthiest federation, as it is able to sell rights to its international tournaments for the most cash. In recent years, this money has been harnessed to develop the footballing infrastructure in smaller European nations. UEFA launched HatTrick, a funding programme for grass-roots development, in 2003; this has part-funded Iceland building over 100 all-weather artificial surfaces for schools. Money can also be decisive if you don’t have it. A lack of cash, together with poor governance, explains why African nations’ preparations for the World Cup have repeatedly been hampered: in the 2014 World Cup, Cameroon, Ghana and Nigeria all suffered financial disputes between their players and boards, leading Ghana’s players to boycott training and Cameroon’s players to arrive a day late in Brazil. “All these institutions need to become much more transparent – Fifa should have a role here. You need German levels of auditing. I would have Fifa auditing and insisting on levels of transparency and withholding money if federations do not comply,” says Goldblatt.

Europe’s institutional advantages are so entrenched that African nations are increasingly dependent on players born and nurtured in Europe. When Morocco last qualified for the World Cup, in 1998, two of their 23-man squad were born outside the country; this time, 17 were. “With the local boys, I’m always working on decision-making,” Mark Wotte, who coaches Morocco’s under-23, told the Financial Times. “On average, the Moroccan trained in Europe is better in every respect than the Moroccan trained in Morocco.”

Queiroz made the same point during the World Cup. “The only nations that are able to compete in the World Cup are those with luck and those who are able to play for clubs in Europe, and that’s not just in Asia but also Africa and elsewhere. People point to Morocco or Senegal as strong African nations, but what they really are is African players playing in Europe.

The World Cup’s expansion, to 48 teams, will increase the likelihood of Africa and Asia going deep into the tournament. Simple numbers explain why: the combined number of African and Asian teams will increase from 10 to 17. For the first time ever, Europe and South American sides will cease to be a majority in the World Cup. And the format – with 16 groups of three, and then 32 teams entering the knockout stages – means there is more chance of weaker teams going further, explains Alex Krumer, a sports economist at Molde University College. The greater importance of the draw, and more knockout games, will simply make the tournament more random. So African and Asian teams should get better results than now, even if they are not any better relative to Europe and South America.

For all the focus on the World Cup, what happens between tournaments is far more important to how nations develop. While a bigger World Cup is welcome for emerging nations, even more important is the need for them to play the best teams in Europe and South America more often. “The more experience the Asian players get, the better they will perform,” says the former Scotland manager Andy Roxburgh, now a technical director for the Asian Football Confederation. “National teams develop by playing regularly against strong opposition.” Roxburgh welcomes Japan and Qatar playing in the Copa América next year.

Yet a short-term boon should not be mistaken for a long-term strategy. If sides from outside Africa and Asia are to enjoy more meaningful matches between tournaments, it will require something altogether more radical to replicate the advantages that smaller nations in Europe and South America enjoy. As Szymanski and Krause note in their paper, “weaker teams from Europe and South America have gained from the continued exposure to top teams and their institutional environment, at the expense of teams from other continents. This calls for stronger integration not only within but also between regions.” For teams from the rest of the world to have the best chance of matching those from Europe and South America, they must play them more in games that matter.

Here there is some hope. Fifa are currently exploring a global nations league – like Uefa’s version but for the whole world. Quietly, this could be a seminal shift: not since 1961, in qualification play-offs for the 1962 World Cup, has an African team faced a country from Europe or South America in a competitive match outside the World Cup. A global nations league would give countries an opportunity to test themselves in between World Cups, to gauge their progress and keep abreast of cutting-edge tactics.

This may represent just the start of greater integration between different federations. In 2016, the Copa América included six teams from Concacaf, a move that may presage full integration of the two confederations, something that the expansion of the World Cup makes more likely. There would be very obvious benefits in a merger: Conmebol’s members would get an influx of cash from broadcasters in Mexico and the US; Concacaf’s teams would be exposed to far more onerous opponents, accelerating their development. Perhaps ultimately a utopian solution is for global qualification with, say, the best 80 teams in the world competing against each other after earlier regional qualifying. This would ensure that one of Europe’s biggest advantages – that its smallest teams get to play against the continent’s best – was shared between all confederations and expose teams’ countries from outside Europe and South America to more global scrutiny, which may improve governance.

Such steps may lessen Europe’s dominance in world football a little. But the continent’s advantages in infrastructure and networks will not easily be eroded. “Europe is the dominant force,” Roxburgh reflects. “Europe has the history, the know-how and the financial power.”

While Europe endures relative decline in other spheres, football is an enclave in which the continent is not merely holding its ground but, in recent years, has actually been strengthening it. And so the promise that 2002 would mark the start of a new, more egalitarian age in international football has been proved half-right: it did, just only for European sides.