Performance-driven culture is ruining scientific research
-
Loading... - SABINA DANIELS
- 16 Feb 2021
- 1127 Views
- 0 Like
- 0 Comment
The first time I heard about the impact factor I was a few weeks into my PhD. A candidate due to finish in a couple of months warned me emphatically: “It makes or breaks careers.” In my innocence, I didn’t think much about it and returned to concentrating on my research. A decade later, metrics such as these came to dominate my work and ultimately drove me to give up my permanent academic post and move into industry.
Since leaving academia, I have found myself wondering about the effect of these metrics on the profession and practice of science.
There is a well-known anecdote about British rule in India. In Delhi, officials were concerned that there were too many cobras. To reduce their population, people were paid for each cobra killed. When the administrators found out that some people had started to breed cobras to kill them and collect the reward, they stopped the scheme. The farmed cobras were set free, causing the population to explode.
This is the so-called cobra effect, which describes how incentives in complex systems can have unintended consequences which exacerbate the problem they were trying to solve.
Academia is a highly incentivised game, and there are great rewards for playing it well: comfortable salaries for those at the top, guaranteed employment for those in the faculty, high-profile appointments, press coverage, shiny medals named after dead scientists and opulent robes to dress up in at graduation. In other words: money, fame and prestige.
There are many good scientists who are driven by the desire to produce meaningful work. But I have also witnessed many who are driven mostly by accolades. Unfortunately, they tend to be in positions of immense power.
They are relatively easy to spot as they seldom speak about their research, only about the politics and strategy of research. They will discuss at great length the ins and outs of performance targets such as impact factor, grant income, invited talks, numbers of research students, editorial positions, board memberships and awards.
In many institutions I have worked at, this thinking has won the day, and ticking boxes was strongly emphasised above everything else. For myself and others, it was made clear that, if we wanted to stay or progress, our sole focus had to be on collecting accolades at all costs.
I think we came to this as a result of the Research Excellence Framework and now the Teaching Excellence Framework. The way these assessment exercises work means that university administrators are increasingly looking to metrics and key performance indicators to compare and be compared against other institutions, and ultimately to secure funding.
But this KPI culture creates a moral danger for the sciences. Their first duty should be clear, objective elucidation of the facts. The effect is easy to see in academic publishing: I have seen co-authors linguistically puff up articles to get them published in prestigious journals, I have reviewed articles that have clearly been inflated, and I have steered research programmes on the basis that a new direction would be easier to publish in high-impact journals.
This ultimately leads to showboat science that under-investigates less eye-catching – but ultimately more useful – areas. The hope is that, with peer review and a culture of professionalism in the sciences, the system is sufficiently self-correcting to inhibit the propagation of outright errors.
I don’t believe this is enough. Many areas of research are so complex that it is difficult and time-consuming to verify or repeat results and – crucially – researchers are discouraged from doing so since it would not have sufficient impact.
I worry that the KPI-driven impact culture increasingly means that careful, meticulous and incremental science is anathema in the academy, especially for those at the early stages of their careers. There are many who are so attracted by the prospect of success that they are willing to obfuscate, mystify and perhaps falsify research to game the system and reap the plentiful rewards.
Most of all, I worry that instead of working towards an enlightened future, many are simply selling farmed cobras and calling it progress.
