Numbers have the power to mislead

Submitted on Wed, 05/31/2017 - 15:05

Viivotin/ Kuva: Jonne Renvall

An international seminar organised at the University of Tampere, Finland, will look at the power wielded by numbers.

Nelli Piattoeva/ Kuva: Jonne Renvall
“Rankings may start to determine and limit our thinking instead of us figuring out what would be the best place to publish our research results,” University Lecturer Nelli Piattoeva says.

An international seminar organised at the University of Tampere, Finland, will look at the power wielded by numbers.

Numbers govern the world. However, numbers are not innocent because they also have the capacity to create misleading impressions of the world around us.

“The problem is that we have started to think that numbers represent reality. Numbers are used to set targets and then determine penalties and rewards. This perverts behaviour, to such an extent that numbers no longer innocently describe reality, but help create it. We have moved to a phase where numbers have become tools of control, and that has affected people’s behaviour,” says Nelli Piattoeva, an adjunct professor in international and comparative education policy who works at the University of Tampere in Finland.

In early June, Piattoeva and Professor Rebecca Boden will organise an international seminar on the power of numbers and related problems.

Boden, who has previously worked as an accountant and a tax official, says that she knows well how numbers can be manipulated so that they appear as people want them to. Boden currently leads the New Social Research Programme at the University of Tampere.

“It is like a game. Most of us encounter such situations in our daily work when we do not do our jobs in the best possible way but in a way that will make the numbers look good,” Boden says.

Apart from individuals, whole organisations in both the public and private sector operate on terms dictated by numbers, figures and one-dimensional statistics.

Numbers may lead
to mismanagement

Rebecca Boden/ Kuva: Jonne Renvall
“Numbers may lead to mismanagement in organisations,” Professor Rebecca Boden says.

Boden is a critical accountant who not only does not see numbers as objective and truthful descriptions of reality but as a means of representing reality in a way that people want to see it.

“The problem is that most people do not understand the power of numbers. When people are given numerical targets, it is deceptively easy and enticing to set goals but these might not equate to the job being done well. For the worker, when they have achieved their numbers, they might treat the job as ‘done’, but it might not be an effective or meaningful job,” Boden says.

Boden takes hospitals as an example; if they can choose to treat only the people with uncomplicated health issues then they can process far more patients and, according to the numbers, they are much more efficient.

“Numbers may lead to mismanagement in organisations,” Boden explains.

Another example Boden is familiar with is the university reform in the United Kingdom. The UK introduced a research productivity measure according to which researchers must produce four journal articles within a four- or a five-year follow-up period – the Research Excellence Framework.

As a result of this new instrument, researchers are disincentivised to use their energy to produce say just one path breaking in-depth research publication, such as a book, within the audit period. Instead, the system encourages them to produce four articles, no more and no less, which might not be the best way to disseminate their work and might lead to numerical productivity at the expense of excellent scholarship.

“We call this salami slicing. It modifies the way researchers do their work and that is not necessarily a good thing,” Boden says.

Weak pupils
are weeded out

In some countries, the tyranny of numbers has hit schools. In Britain, schools are pressured to have all 16-year-olds graduate with good GCSE grades in five subjects. Schools are punished if they do not fulfil this objective regardless of the pupils’ background or learning challenges.

“Instead of offering a good education to the pupils and making them good citizens, schools are often concerned only with their school-leaving exams. That means teaching for the exams and tests. The pupils pass their exams but that does not mean that they have received a good education,” Boden says.

Piattoeva, who knows the Finnish school system, says that Finland does not yet have similar problems as the UK. With Galina Gurova and Tuomas Takala she has studied quality assurance and evaluation in Russian schools where the problems resemble those in the UK.

“In Russia, schools are rewarded or penalised on the basis of whether they are able to reach predetermined learning outcomes measured by compulsory national tests. Based on the test results, schools are publicly ranked and allocated resources, and recruitment decisions are also based on these outcomes. Schools may even expel pupils who endanger their school’s statistics with poor performance,” Piattoeva says.

The system jeopardises the children’s opportunities to get a good education and their equal opportunities for further study. Pupils who need extra help are not always offered assistance even though that would help them complete their degrees.

In Russia, United Kingdom and the United States, the school system marginalises and disadvantages the weakest students and those facing the greatest difficulties.

“In Britain, this problem is seen with children with immigrant or Romany backgrounds and those pupils who have learning difficulties or behavioural and health problems,” Boden says.

Rankings dictate
the way researchers work

The power of numbers increasingly affects the work researchers are doing because publication forum classifications determine how universities are allocated resources.

Piattoeva says that she has heard researchers and administrators discuss how important it is to publish in publications that belong to the highest publication forum classification.

“The ranking of the publications is starting to dictate the researchers’ decision-making instead of us figuring out for ourselves which journal would be the best for which article. This is how numbers are encroaching on our thinking and everyday lives,” Piattoeva says.

“We are doing this to ourselves. If you look at a Google Scholar profile, you can see how many citations our publications have amassed and what our indices are. If you do not beware, you may start to evaluate yourself according to your own numbers,” Boden warns.

The public sector
has adopted business models

Leading by numbers comes to the public sector from the private sector where business success is determined by monetary profits. The public sector is not driven by the ‘bottom line’ of profit, so audit firms and management consultants have instead encouraged the use of non-monetary measures of success, such as performance targets.

Boden finds it problematic that public sector performance is evaluated using such indicators, derived from profit-seeking businesses, even though the public sector does not need to make a profit. The targets are invariably quantitative, e.g. how many pupils pass through the school system and achieve the designated number of exam passes. Boden finds such systems disturbing and detrimental.

“Governments in both Britain and elsewhere put constant pressure on research. They do not understand qualitative indicators. Instead, they want to see numbers and hard facts. The numbers define the whole research agenda, and it is very hard to make counter-arguments,” Boden laments.

Researchers’ duty is to expose
the number-based lies

According to Piattoeva, especially young researchers are very vulnerable to the power of numbers.

“You may be concerned and critical but if you want to feed your family and secure your future you must find a balance. Researchers cannot resist the dominant thinking. Instead, the question is of a continuous negotiation about how far you are willing to go and how much you want to preserve your own genuine and original goals. You must be honest to yourself but at the same time you should not burn your bridges either,” Piattoeva says.

Boden says that it is the researchers’ job to expose and explain how loosely the numbers are produced and who controls them.

“Who writes these rules and who has the responsibility for the schools that push the weaker students away? We have to find the stories behind the numbers, and researchers are good at that,” Boden concludes.

Text: Heikki Laurinolli
Photographs: Jonne Renvall

Escaping numbers? The ambiguities of governance through data in public sector. The seminar is organised at the University of Tampere on 2 June 2017.