Publishing and perishing under gameable metrics 2

This article about the Australian Excellence in Research for Australia (ERA) initiative discusses how the process by which Australian universities and academic are assessed is flawed. It also discusses how Australian institutions have been gaming the metrics, like certain New Zealand institutions have been accused of doing.

In this previous post I described how any metric by which an institution or academic is assessed can be gamed. That is, any way in which an academic or institution is assessed can be manipulated by that institution to gain a higher score. In this post, I discussed how this has a negative effect on the teaching performance of an institution. By removing staff who do not perform well in research assessments due to a heavy teaching load, the institution can lift their research scores, but at the cost of lowering their teaching performance. As the article mentions, teaching is not assessed, so the process optimises towards a single metric at the expense of all others. This is not helpful for the long-term viability of an institution, as undergraduates will not want to attend an institution with a poor reputation for teaching.

This situation is almost certain to increase the use of contract lecturers, as contract lecturers are, as I understand it, exempt from assessment. I've already described why increasing contract lecturers is a bad idea, mostly because of a lack of job security and satisfaction for the contract lecturers, as well as a lack of continuity in teaching from the point of view of the students.

It is becoming increasingly apparent to me that assessing institutions is not as useful as assessing individuals, and that, in today's highly-mobile world, the reputation of an institution is no longer as important as the reputation of an individual researcher. This raises an interesting question:

What would happen if research performance based funding were given directly to the researchers based on their own individual performance, rather than their institutions being given extra funding based on the collective research performance of their staff?

The article linked at the start of this post does an excellent job of describing the problems with collective assessments (like what does it mean if you have one researcher ranked 1 and one ranked 5 - do they have a collective performance of 3? What does that even mean?).

Individual funding would remove a lot of the financial motivation for institutions to game the system, although it wouldn't eliminate it (institutions would still make money by charging the individual researchers over-heads, but these could be capped). Under the current Australian and New Zealand systems, individuals are assessed anyway, so it doesn't require any great changes to the current assessment process. One downside (and it could be a stonking big downside) is that early-career researchers would probably do poorly under this model. Early-career are already disadvantaged by management practices designed to game the system, and a simple weighting mechanism accounting for the length of time an individual has been doing research would go a long way to help. This would encourage researchers to start publishing early (which is essential to master the art of scientific publishing) and to publish consistently (which is essential to maintain your publishing skills). Another downside would be senior researchers taking credit for the work of junior researchers. But, again, this happens anyway, even though it is profoundly unethical. Under this system, though, it would no longer be just unethical, it would be criminal fraud.

Such a scheme could only be successful if it were paired with a scheme for assessing and rewarding teaching. While I have stated several times that an academic in a permanent position who is not publishing is not doing their job, an academic with a low (but not non-existent) research output and a strong teaching performance is an asset to an institution. Therefore, it is, in my opinion, imperative that an objective metric for teaching performance be implemented as soon as possible. That way, quality teachers, as well as quality researchers, would be recognised and rewarded. Those who do both (and this is the ideal for an academic, to teach and do research) would score even higher.