The problem with academic journals 6

In my previous posts on academic journals (see here, here, here, here, here, and here) I've discussed the major problem with academic journals in the context of the huge cost of accessing the content that the journals receive for free, as well as the importance of open-access journals. This post is concerned with another problem that is becoming apparent with journals: the declining acceptance rate for papers submitted to journals, in attempts to foster an image of exclusivity and quality.

A recent editorial by David Wardle describes a quantitative analysis he performed that compared the acceptance rates of four top-ranked ecological journals with the large open-access journal PLoS One, along with the citation rate of papers published in each. What he found was that the four traditional journals accepted less than 20% of the paper submitted to them, while PLoS One accepted around 69%. However, papers that are published in PLoS One are cited more than papers published in one of the traditional journals. His argument was that the traditional journals rejected papers that were of good scientific quality (that is, they described good work) but were not "worthy" of publication in such "august" journals, with the editors using the excuse that limited page space meant that there wasn't room to print the papers, even though they were quite good. He then goes on to explain that this exclusivity was motivated by a desire to increase the perception of quality of the journals. That is, the editors are trying to foster the impression that the journals must be really good, because they're really picky about which papers they publish.

But, the ultimate measure of the quality of a paper is how often it is cited, as that reflects how useful it is to other scientists, and papers published in the less-exclusive open-access journals are cited more. Thus, the concept that journals with low acceptance rates publish better papers is fatally flawed: these journals are rejecting papers that are scientifically sound and are useful to other scientists.

This leads me to think that the only reason the top journals are the top journals are because people think they are. If someone wants an authoritative citation to back up a statement they make in a paper, they will cite a paper in Nature or Science if they can, because these are the top journals (this doesn't happen much in computational intelligence, because very few papers in this field are published in Nature or Science). But the conclusion of Wardle's study is that acceptance rate is not a reliable metric of the quality of a journal. If anything, it is a measure of the snobbery of a journal.

The purpose of peer review (and of reviewers) is as a crap-filter for papers, to keep work that is incorrectly done or poorly presented from entering the literature. But with exclusive journals, the peer reviewers seem to be spending more time deciding which papers are significant enough to be published in the journal, rather than trying to identify flaws in the work. The whole thing reminds me of the reason the great physicist Richard Feynman quit the US National Academy of Science: because they spent most of their time deciding who was "worthy" of joining the Academy.

Not so long ago, we had to consider the quality of journals because it wasn't feasible to track the impact of a single paper. Now, with tools like Google Scholar, we can track the citation histories of individual papers. In short, the journal in which a paper is published is no longer that important: the usefulness, the contribution of the paper is what is important. By the same token, the quality of an academic is not measured by which institution they work for, but by their contributions. Unfortunately, the bean-counters who make the hiring and promotion decisions, and who make decisions on who gets competitive research funding, haven't grasped this concept yet.

Exclusive journals do not make a good contribution to science, as they keep too much useful material out of the public eye for too long: peer-reviewed open-access journals, with their more liberal acceptance rates, are more important then ever in this situation.