Do global ranking lists and surveys actually reflect performance?

8:49 pm on 5 December 2018

By David Cohen*

Opinion - Exciting news. The Legatum Institute has just released its 12th annual global Prosperity Index, a hefty survey that ranks 149 countries by "prosperity".

Hand with chalk is drawing Ranking concept on the chalkboard.

New Zealand has been ranked second overall in the Legatum Institute's gobal Prosperity Index. Photo: 123RF

Once again this year New Zealand does notably well - as indeed we seem to in all of these exercises other than perhaps the one that really counts.

The London-based think tank's new rankings consider 100 variables having to do with prosperity, including economic strength, workforce participation and the natural environment, which are then broken down into nine categories.

This country comes out top in the environmental stakes, helping it to be named second-best overall.

The Legatum exercise comes on the heels of another perky survey, by the Spectator Index website, showing New Zealand as the world's second-most peaceful nation.

The Spectator - not to be confused with the British masthead of the same name - is one of many media enterprises that spit out a seemingly unending stream of statistics and lists these days for the benefit of news consumers, not least in New Zealand.

In addition to the news that New Zealand is second only to Iceland in the tranquillity stakes, for instance, the country apparently has the world's sixth-most reliable police services, according to the Spectator.

We have recently been named the world's best place to visit, the best place to be an expat or to do business, the best in the reputable stakes and the second-best for all-round experience.

Not to mention - listen up, accused parliamentary bullies - the third most-trusted politicians on the planet, according to the Spectator.

Another media survey, drawing on the views of experts and "informed elites" has New Zealand at, well, number 14.

And still the hits keep rolling.

The Guardian had one of the "best" names for these best-of lists - listicles, the British masthead calls them, and while they bring cheer to at least some news consumers, they're not necessarily a good sign for the media business.

Rather than revealing anything terribly important about various nations, these items tend to be more about page impressions, "a factor that drives advertising revenue, which has increased speculation that some listicles are commercially - rather than content - motivated," the outlet notes.

Interestingly, if not worryingly, one such regular listing that New Zealand never seems to do so well in is the one that the country probably invests the most public money into acing: the various media rankings of the world's "best" universities.

In one of the latest, the QS World University Rankings, the country's eight universities all scored places, with Auckland even cracking the top 100. Most of the others moved down the Top 1000.

More recently, in the rival list produced by Times Higher Education, no local universities cracked the top 200 at all.

Nevertheless, according to Universities New Zealand chief Chris Whelan, such surveys remain an important bellwether for governments to pay attention to - which is to say, invest public money in.

But are they? Part of the reason the results in the QS survey and others like it change annually is because those who conduct the surveys simply vary the process from year to year, creating arbitrary shifts that have little to do with actual performance.

The same listings also tend to use things like student/staff ratios as a proxy for teaching quality, or give 20 percent to research and 20 percent to reputation, which can be seen as arbitrary at best or even meaningless.

"There's no theory at the base of this," the Australian scholar Simon Marginson, a professor of international higher education, has tartly observed. "It's just people sitting in a market research company and a media office, guessing about how to best manipulate the sector.

What to do? Perhaps the "best" response to any of these surveys, whether bringing news that New Zealand is in at number 900 with a bullet among the world's academic institutions or is the leading nation for blind dates, is to reach for a world-beating brand of salt and mix it with the latest news.

*David Cohen is a Wellington-based journalist and writer.

Get the RNZ app

for ad-free news and current affairs