The problem with academic journals 8

It's been a long time since I last blogged about the problems with academic journals. Several of my old posts described the behaviour of the giant academic publisher Elsevier, specifically trying to buy a law in the US Congress that would virtually ban researchers publishing in open-access journals. This resulted in an enormous backlash against Elsevier, including a boycott that now has more than 14,000 names, culminating in the proposed legislation being dropped.

Unfortunately, Elsevier is back to their old bad behaviour: they have been sending notices to researchers and academic network sites demanding the removal from the web of papers that the researchers' had published in Elsevier journals. While Elsevier may be within their legal rights to do so (since they demand that authors sign over copyright to Elsevier), preventing people from self-archiving papers that they wrote is highly
 detrimental to science. In other words, Elsevier gets the research papers for free (submitted by the authors), they get the quality control for free (done by volunteer reviewers), and the administration of journals for free (done by volunteer editors). Then, they do some basic formatting and proof-reading, demand that the authors surrender all rights to the article, and publish it at an enormous profit.

Some publishers like the IEEE work the same way but allow for self-archiving, that is, they allow authors to post papers they have authored on their own websites for other researchers to access. The IEEE seems to be doing quite well out of this practice, but then the high offices of the IEEE are held by engineers and academics rather than businessmen. Does Elsevier really think that they can get away with this kind of bully-boy behaviour?

There are a couple of Elsevier journals that I've published several papers in, and I still have research that I was going to submit to them. But now I think that It's time for me to find some alternative journals to submit my work to. I'm currently reviewing one article for an Elsevier journal, and I took that task on because a friend asked me to, but after that, I won't review for any Elsevier journals. And I will not, under any circumstances, serve on the editorial board of any Elsevier journals.

Video: How to publish your research in IEEE CIS publications

A panel discussion on how to publish in the IEEE Computational Intelligence Society's publications. This discussion wraps up the individual talks by Professors Xin Yao, Kay Chen Tan, Derong Liu, Garrison Greenwood, Chin-Teng Lin and Simon M. Lucas . This panel session was part of the CEC 2013 conference.

How to publish your research: Video of Professor Simon M. Lucas

Professor Simon M. Lucas talks about how to publish research in IEEE Transactions on Computational Intelligence and Artificial Intelligence in Games. This talk was part of a panel session at CEC 2013 and mostly talks about what kind of papers are published in TCIAIG.

How to publish your research: Video of Professor Chin-Teng Lin

Professor Chin-Teng Lin, who is the editor-in-chief of IEEE Transactions on Fuzzy Systems, speaks about publishing in that journal. This talk was part of a panel discussion at the CEC 2013 conference. Some of the points he makes in this talk are applicable to publishing in most journals:

• Read existing papers, know the field
• Present an issue of significance
• Choose a journal that fits well with your research
• Use the correct format for that journal
• Have focus & vision, don't be too ambitious with your paper
• Write clearly
• Get pre-review, ask  your colleagues to check your paper before submission
• Proofread!
• Be patient, reviews take time

How to publish your research: Video of Professor Garrison Greenwood

Professor Garrison Greenwood talks about how to publish your research in the IEEE Transactions on Evolutionary Computation. This talk was part of a panel session at the CEC 2013 conference.

The main points of this talk are:

• Establish the context of your research in the Related Research section following the introduction (as a side-note, in other fields this is the introduction: why does computer science insist of separating them?

• Use enough citations, and no more

• Write enough detail for a competent researcher to replicate your work, don't try to write a tutorial

• Read the instructions for authors

• Use good grammar and spelling

How to publish your research: Video of Professor Derong Liu

Professor Derong Liu talks about how to publish your research in IEEE Transactions on Neural Networks and Learning Systems at a panel session at the CEC 2013 conference. This talk gives a lot of information about the editorial and review process, as well as how to increase your chances of having a paper accepted and even how to get on the editorial board of the journal.

How to publish your research: Video of Professor Kay Chen Tan

A video of a talk by Professor Kay Chen Tan as part of a panel session at the CEC 2013 conference. In this video he describes the IEEE CIS Computational Intelligence Magazine.

How to publish your research: Video of Professor Xin Yao

This is a video of a talk given by Professor Xin Yao as part of a panel session at the CEC 2013 conference. A couple of the most salient points that I noticed:

1) If you want to publish your research, you must first do good research
2) Contact the editors of the journals you want to publish in before submitting

Down with middlemen!

If there is one thing that the rise of e-commerce sites like eBay and Amazon.com has shown it is that the middleman is doomed. Entire bookshop chains like Borders have vanished from the face of the Earth, largely because they were unable to compete with a model that has no physical presence. Bookshops are middlemen: they connect one group of people (publishers) with another group of people (book purchasers, consumers). And the one thing that the Internet is really good at is getting rid of middlemen. Even publishers are middlemen, they don't produce the product, the authors do. The rise of self-publishing is the strongest indicator yet that the publishers are, like bookshops, an endangered species.

Online sales sites like e-bay (and Trademe, Gumtree) have also had an impact on retailers, and second hand dealers in particular: when we moved from Australia back to New Zealand, we had to sell our car and a few others bits and pieces. We didn't take the car to a second hand car dealer, or call a second hand furniture shop about our excess furniture: we just put some adverts up on Gumtree. While it is harder for online retailers to compete on items that require large volumes such as groceries, for smaller-volume or speciality items online sites are slowly but surely eliminating the traditional merchants. In the last ten years the only time I've booked air travel through a travel agent was for business travel, and then only because my employers had a policy of booking through certain agents.

This all raises a question: what is a middleman? If we define middlemen to be someone who does not produce, add value or provide a service that cannot be automated, then a huge number of current professions come under that heading: real estate agents, immigration agents, literary agent, property management agent... Basically, anyone with the word "agent" in their job title is a middleman and is doomed.

How does this relate to computational intelligence or academia? Well, what if journals and universities are really middlemen?

In the past I have blogged about how open-access journals are the future of academic publishing. But how much value do journals of any kind really add? A journal will arange peer-review, format the accepted articles and assign volume/page/DOI numbers. Apart from peer-review, each of these steps can be automated. In an age when every article published is available online, and are indexed by sites like Google Scholar and Citeseer, journals don't add much to the publicity of an article - in fact, the most effective way of publicising an article seems to be to blog or tweet about it. This is still the major advantage of open-access journals, as anyone with an interest can download and read the article (and hopefully cite it).

The measure of the quality of an article is the number of citations it receives, much more so than the supposed quality of the journal it is published in. Metrics like impact factor are so bogus as to be meaningless, despite the arrogant attitude of editors who deem submissions unworthy of publication in their august journal, without bothering to send them to peer-review. A good article will be cited more, no matter where it is published. Articles that aren't useful won't be cited. In other words, articles now can stand on their own, they don't need the support of journals to be useful. The journals, therefore, are middlemen, standing between the producers (the people who do the research and write it up) and consumers (the people who are reading and citing the research). Do we really need journals to arrange peer review? Or is there scope for a journal-agnostic, peer-review service for individual articles?

If individual articles can now stand on their own, how about individual academics? The Khan academy has been described as a revolution in teaching numerous times, and open courses like those offered by MIT have had thousands of students. In many ways universities are middlemen, providing access to resources (academic staff) to consumers (students). Universities provide tuition, consultation (students can ask their instructors for clarification), assessment (tests, assignments and exams), and accreditation (a degree / diploma from an institution has a certain credibility). Tuition can be supplied directly by the lecturer via sites like YouTube. Consultation can be done via discussion boards and live chat. Accreditation remains as an open problem. There are a huge number of accreditations available in a vast range of technical subjects: the IT industry in many ways leads the way in this, with certifications from Microsoft, Cisco, CompTIA and others. Professional organisations like the IEEE publish bodies of knowledge that graduates in certain disciplines are expected to know, and it's only a matter of time before this is expanded to include computational intelligence. Practical work is harder to deal with, but even then the large amount of open source software available means that anyone with a cobbled-together Linux box and a basic internet connection can not only do the lecture and practical work associated with undergrad study but also access the accreditation offered by numerous organisations.

The only problem for which I cannot see an obvious solution is, how would the lecturers get paid? Locking material behind paywalls won't work, people just won't use it. Also, a fixed fee won't work either: $500 might not seem like much for someone in the western world, but for someone in parts of Africa, it's more than they see in a year. The pay-what-you can model might work: this is where someone pays as much as they think something is worth, or as much as they can afford. A few people might take advantage and pay nothing when they could afford to pay, but most people are pretty honest and will pay a fair price. The accreditation agencies could also pay a referral fee to lecturers who direct students to their services, much like the Amazon affiliates program.

Universities would still survive, there still needs to be places where research is carried out, and training of the next generation of researchers (postgraduate students) takes place. The survival of journals is a bit less certain, as self-published peer-reviewed articles are much easier to do. Whatever happens, though, middlemen are on the way out.

Guest post: Write Right First Time with Brown's Eight Questions

This is a guest post by Stephen G. Matthews. Stephen is a PhD student in the Centre for Computational Intelligence at De Montfort University, UK.

---

Write Right First Time with Brown's Eight Questions

I will share a method that I have found to be really useful. It's short, simple and incredibly effective: Brown's Eight Questions.

Robert Brown introduced Brown's Eight Questions (Brown, 1994/95) as part of an action learning set for improving writing. An action learning set is a group of people (ideally 5) who meet up to discuss common problems and solutions. Brown suggests applying this to writing for publication. An action learning set meets up and each member reviews each other's manuscripts face to face. I will focus on Brown's Eight Questions, but an action learning set for writing is well worth reading about in Brown's article.

So what is Brown's Eight Questions? Well, it is a set of eight questions designed to make an author think about writing before actually writing a first draft. Brown's idea, which was motivated by his experiences as a writer, reviewer and editor, comes from his observation that writers often focus on correcting a manuscript once it is written, rather than planning the manuscript before writing.

Brown’s Eight Questions

1. Who are the intended readers? - list 3 to 5 of them by name;
2. What did you do? (limit - 50 words)
3. Why did you do it? (limit - 50 words)
4. What happened? (limit - 50 words)
5. What do the results mean in theory? (limit - 50 words)
6. What do the results mean in practice? (limit - 50 words)
7. What is the key benefit for your readers? (limit - 25 words)
8. What remains unresolved? (no word limit)

As you can see these are simple questions and the word limit ensures answers are succinct. Brown's Eight Questions can help authors to gain clarity about their manuscript and also support the reader's understanding.

Brown's Eight Questions helps me to structure my thoughts, arguments and the message of a manuscript. It really is a useful method that can be applied to any form of writing such as journal articles, theses and reports. If you have not used it then give it a go!

Brown, Robert (1994/95) “Write Right First Time”, Literati Newsline Special Issue: 1-8. (Available from http://web.archive.org/web/19971014014626/http://www.mcb.co.uk/literati/write.htm)

The problem with academic journals 6

In my previous posts on academic journals (see here, here, here, here, here, and here) I've discussed the major problem with academic journals in the context of the huge cost of accessing the content that the journals receive for free, as well as the importance of open-access journals. This post is concerned with another problem that is becoming apparent with journals: the declining acceptance rate for papers submitted to journals, in attempts to foster an image of exclusivity and quality.

A recent editorial by David Wardle describes a quantitative analysis he performed that compared the acceptance rates of four top-ranked ecological journals with the large open-access journal PLoS One, along with the citation rate of papers published in each. What he found was that the four traditional journals accepted less than 20% of the paper submitted to them, while PLoS One accepted around 69%. However, papers that are published in PLoS One are cited more than papers published in one of the traditional journals. His argument was that the traditional journals rejected papers that were of good scientific quality (that is, they described good work) but were not "worthy" of publication in such "august" journals, with the editors using the excuse that limited page space meant that there wasn't room to print the papers, even though they were quite good. He then goes on to explain that this exclusivity was motivated by a desire to increase the perception of quality of the journals. That is, the editors are trying to foster the impression that the journals must be really good, because they're really picky about which papers they publish.

But, the ultimate measure of the quality of a paper is how often it is cited, as that reflects how useful it is to other scientists, and papers published in the less-exclusive open-access journals are cited more. Thus, the concept that journals with low acceptance rates publish better papers is fatally flawed: these journals are rejecting papers that are scientifically sound and are useful to other scientists.

This leads me to think that the only reason the top journals are the top journals are because people think they are. If someone wants an authoritative citation to back up a statement they make in a paper, they will cite a paper in Nature or Science if they can, because these are the top journals (this doesn't happen much in computational intelligence, because very few papers in this field are published in Nature or Science). But the conclusion of Wardle's study is that acceptance rate is not a reliable metric of the quality of a journal. If anything, it is a measure of the snobbery of a journal.

The purpose of peer review (and of reviewers) is as a crap-filter for papers, to keep work that is incorrectly done or poorly presented from entering the literature. But with exclusive journals, the peer reviewers seem to be spending more time deciding which papers are significant enough to be published in the journal, rather than trying to identify flaws in the work. The whole thing reminds me of the reason the great physicist Richard Feynman quit the US National Academy of Science: because they spent most of their time deciding who was "worthy" of joining the Academy.

Not so long ago, we had to consider the quality of journals because it wasn't feasible to track the impact of a single paper. Now, with tools like Google Scholar, we can track the citation histories of individual papers. In short, the journal in which a paper is published is no longer that important: the usefulness, the contribution of the paper is what is important. By the same token, the quality of an academic is not measured by which institution they work for, but by their contributions. Unfortunately, the bean-counters who make the hiring and promotion decisions, and who make decisions on who gets competitive research funding, haven't grasped this concept yet.

Exclusive journals do not make a good contribution to science, as they keep too much useful material out of the public eye for too long: peer-reviewed open-access journals, with their more liberal acceptance rates, are more important then ever in this situation.

More on open access journals

Continuing my series of posts on open access journals (see here and here), this article by Simon Owens in U.S. News is an excellent and detailed review of the debate. The article compares open access journals to e-books: while e-books have existed for a long time, it is only in the last five years that they have really taken off, after reaching a tipping point. Owens argues that open access journals have reached that tipping point, and the academic journal publishing business (known for the huge profits they extract from university libraries) is on the verge of serious disruption.

I tend to agree with his assessment, open access journals have been flying largely under the radar for a long time, but I get the sense that they are becoming more accepted among the top researchers: when more top researchers publish in open-access journals, they will gain credibility.

The old publishing model is being destroyed by greed: journals are just too expensive, and suck too much money out of universities that should be spent funding research and paying people's salaries. Open access is the future of scientific publishing.

A small victory for open access 2

Following up from my earlier post, this article in The Economist gives a pretty good overview of the recent UK and EU move towards requiring the outputs of publicly-funded research being published as open access. The article also gives a lot of context about the different open access publishing models - the "gold" model practiced by PLoS, where authors pay a fee to publish; and the "green" model that the USA's NIH demands, whereby papers are published in traditional journals, but the journals must allow authors to publish their papers in an open repository like PubMed after one year.

So, when are we going to start seeing one of these models applied to computational intelligence journals? I'd be especially pleased if the IEEE were to adopt one of these models, as they lock every single paper they publish up behind a paywall, seemingly for all of time.

A small victory for open access

All taxpayer-funded research in the UK must now be published as open access papers, according to this article in the BBC. The British government will be providing £50m in subsidies for researchers to pay the fees necessary to have their work available as open access.

This is a victory for open access. But, the victory is not complete. Firstly, the £50m is coming out of general research funding, it's not new money. In other words, there will be less research done because of this, as there will be less money available to fund it. Secondly, the money is going to the established academic publishers, who are just going to use it to further pad their profits. Finally, as the article states, many journals will still not accept articles that have the relevant data available from open data repositories.

I still think that eventually, open access journals will over-whelm the old publishers. But they can only do this if the top researchers contribute quality research articles to them. Meanwhile, I personally think that the next step is for reviewers (and editors) to start demanding payment for the labour they provide to the publishers. It is we reviewers and editors who provide the quality control for the journals, it's time we got paid for it.

Would anyone be willing to sign up for a boycott of all publishers, until reviewers and editors are paid?

More developments in academic journals

There has been a new development in open-access academic journals (see my previous posts on this matter here, here, here, here and here). Two articles, here and here, describe PeerJ, a new approach to open-access journals. Whereas the traditional publishers charge readers for access to content, and open-access journals charge authors per-paper publication charges, PeerJ charges authors a one-off lifetime publishing fee. As long as all of the authors (or at least the first 12 authors) of a paper are subscribers, the authors can submit as many papers as they like for no further cost. The papers are peer-reviewed, and will be available for free. There are different subscriptions available, ranging from a lower-cost option that allows for a fixed number of papers per year, up to a more expensive "all you can eat" model with no restrictions.

PeerJ is starting with life sciences first: given the large number of researchers and papers coming out of the life sciences, this seems quite sensible and is more likely to give them a solid revenue stream early-on. It is interesting that they are requiring each member to review at least one paper per year, which neatly gets around the problems associated with finding enough reviewers for papers.

I suspect that the computational intelligence community does not have enough researchers to make such a model viable at the rates PeerJ are advertising. So, such a journal would probably have to charge higher subscription rates, or charge an annual or bi-annual fee.

But these are all ways for publishers to make money off of free content (submitted papers) and free labour, in the form of reviewers (who are actually paying for the privilege in the case of PeerJ). I'm not the first person to suggest this, but why not spend some of that money on reviewers? That is, when a reviewer completes an on-time review, pay them a small gratuity (like 100-200 Euros). That would motivate reviewers to do their reviews on time (if you're working for free, there is less motivation to do the work quickly). It would also be a more fair system, as those who provide the most valuable service in the publishing process would be compensated for their time and efforts. Finally, it might make it easier to find reviewers for papers: my own editorial experience has shown me how hard it can be to find reviewers for a paper. I review about a dozen papers per year, so this scheme wouldn't provide me with a living, but it would cover many of the incidental expenses that come up over the year.

Instead of a Boycott Elsevier pledge website, do we need a website where people can pledge to no longer review any papers until publishers start paying? Would anyone sign up for that?

An experiment in open-source textbooks

I am thinking of writing a textbook. Actually, I'm working on three at the moment, one of which is a research monograph, but the one that it most relevant to this post is tentatively titled Intelligent Information Systems, and will cover neural networks, fuzzy systems and evolutionary algorithms at an undergraduate level. I also expect it would be useful for researchers from other disciplines who want to apply methods in computational intelligence to their own research, and to software engineers who want to solve real-world problems with computational intelligence.

In line with this post, I am seriously considering making Intelligent Information Systems available as an open-source textbook. But before I do, I need some encouragement. So I'm asking you, my dear readers, to encourage me. If you think you would assign an open-source textbook on this topic to a class, or that you would buy a self-published textbook, let me know in the comments. If you could see yourself contributing some other way, let me know, too.

It's up to you good folk to push me to do this!

The problem with academic journals: An update

 A brief update on the status of the Elsevier boycott (described here): to date, more than 11 000 academics have pledged to not review, submit or do editorial work for any Elsevier journals. My previous post has already described why I oppose such a boycott of a single publisher, and I expect that this boycott is going to cause some unanticipated consequences.

I suspect that this boycott explains why the papers I have under review in Ecological Modelling and Ecological Informatics are taking so long to go through the review process: it's hard enough finding reviewers as it is, and with people refusing to review for Elsevier, it's going to get even harder. That's not punishing Elsevier, that's punishing the researchers who are trying to get their work published and advance their careers.

As I said before, the way real change will come about is by the top researchers supporting open-access journals. At least one of the people who could do this has just done so: Winston Hide, an associate editor at the highly-ranked Elsevier journal Genomics has just resigned from the editorial board, with the avowed intention of focusing his energies on open-access alternatives. I can only hope that some of the top researchers in computational intelligence will do the same.

Open source textbooks

The principle of Open Source Software (OSS) has been established for a long time. The Linux OS (or GNU/Linux, for the purists out there) made the idea of freely giving away software and source code respectable. Concerns about the quality of the software, and whether or not companies could make money from open source, have all been washed away over the years. OSS tends to be more stable, has bugs fixed faster, and evolves faster than commercial software. Also, companies have been making money from OSS for years: Red Hat being just one example.

In this pod cast transcript Steven Cherry from IEEE Spectrum talks with Richard Baraniuk of Rice University about Open Source Textbooks (OST). Baraniuk has founded the Connexions platform, a platform for developing open source textbooks.

I can think of several objections to the idea of OST, but I believe that, in common with OSS, these objections are not insurmountable problems:

Firstly, there is the issue of quality control. When an author submits a book proposal to a publisher, the publisher will send the outline and a sample chapter to reviewers. But, the reviewers tend to be people the author knows, as unlike anonymous peer review of journal articles, a textbook author can often nominate the reviewers of their proposal.

Secondly, there is the issue of formatting the book. If you use an authoring system like LaTeX, formatting a book isn't really that hard (certainly easier than formatting a book in Word). Publishers tend to only provide an author with a template, anyway.

Thirdly, advertising the book. This seems to vary fairly widely between different publishers, with some putting a lot of effort into it, and others doing much less. With the reach that the Internet provides people now, I don't see advertising as a large issue. If you have a blog, website, or networking profile (and I think that a serious academic should have all of these), you can advertise your book there. If you can afford it, you can buy some ads through Google or one of the other advertising services. It takes a bit of work, but not as much as writing the book in the first place.

Fourthly, producing the book. If you are going entirely for a soft-copy, open-source approach, that's not a problem: just whack the book up on a website, and let people download it. If you want to sell hard copies, then you can go with a publishing-on-demand (POD) service like lulu.com. Using POD has the advantage that you don't need to pay for inventory before you can start selling copies. That is, while most traditional publishers like to produce the hard copies themselves, they also like to print several thousand copies, and then sell them. With POD, copies are printed as they are sold. No inventory, so no big pile of books (money!) sitting in a warehouse where they might get sold later on. If the publisher doesn't decide to kill the book, or sell the lot off at a loss, or just pulp them.

Finally, money! Traditional publishers take a big chunk of the sale price of a book for themselves: around 90%, or more. Combined with the relatively small number of copies that most textbooks sell, an author isn't going to make a lot of money from the exercise (there are exceptions, but it's a pretty long tail: most textbook authors will make very little money, and just a few will make a lot). If you publish open-source, then there are other ways of making money from the book - advertising on the website you host it on, soliciting donations, and selling hard copies via POD services, which tend to give larger shares to the authors. For an early-career author like myself, the biggest problem I face isn't missing out on a royalty cheque, it's obscurity.

I've come to realise that, in common with the problems with academic journal publishers, textbook publishers really don't add that much value. Sure, there is the cachet associated with publishing with certain publishers, just as there is with publishing with certain journals, but is that enough of a reason to put up with their disadvantages?

An OST system like Connexions also solve most of the objections I listed above: material that is submitted to Connexions is subject to peer review, it is becoming well-established as a place to go to for OST, and they sell hard copies. I really do think that, just as open access journals are the future for publishing papers, open source textbooks are the future of textbooks, and that within a generation (certainly within my working lifetime) we will see traditional text book publishers diminish in importance.

Is Connexions to OST as Source Forge is to OSS? Would you spend money to buy a hard-copy of an OST textbook? Would you contribute money in other ways to support the work of an OST author? Would you assign an OST as a class textbook?