Down with middlemen!

If there is one thing that the rise of e-commerce sites like eBay and Amazon.com has shown it is that the middleman is doomed. Entire bookshop chains like Borders have vanished from the face of the Earth, largely because they were unable to compete with a model that has no physical presence. Bookshops are middlemen: they connect one group of people (publishers) with another group of people (book purchasers, consumers). And the one thing that the Internet is really good at is getting rid of middlemen. Even publishers are middlemen, they don't produce the product, the authors do. The rise of self-publishing is the strongest indicator yet that the publishers are, like bookshops, an endangered species.

Online sales sites like e-bay (and Trademe, Gumtree) have also had an impact on retailers, and second hand dealers in particular: when we moved from Australia back to New Zealand, we had to sell our car and a few others bits and pieces. We didn't take the car to a second hand car dealer, or call a second hand furniture shop about our excess furniture: we just put some adverts up on Gumtree. While it is harder for online retailers to compete on items that require large volumes such as groceries, for smaller-volume or speciality items online sites are slowly but surely eliminating the traditional merchants. In the last ten years the only time I've booked air travel through a travel agent was for business travel, and then only because my employers had a policy of booking through certain agents.

This all raises a question: what is a middleman? If we define middlemen to be someone who does not produce, add value or provide a service that cannot be automated, then a huge number of current professions come under that heading: real estate agents, immigration agents, literary agent, property management agent... Basically, anyone with the word "agent" in their job title is a middleman and is doomed.

How does this relate to computational intelligence or academia? Well, what if journals and universities are really middlemen?

In the past I have blogged about how open-access journals are the future of academic publishing. But how much value do journals of any kind really add? A journal will arange peer-review, format the accepted articles and assign volume/page/DOI numbers. Apart from peer-review, each of these steps can be automated. In an age when every article published is available online, and are indexed by sites like Google Scholar and Citeseer, journals don't add much to the publicity of an article - in fact, the most effective way of publicising an article seems to be to blog or tweet about it. This is still the major advantage of open-access journals, as anyone with an interest can download and read the article (and hopefully cite it).

The measure of the quality of an article is the number of citations it receives, much more so than the supposed quality of the journal it is published in. Metrics like impact factor are so bogus as to be meaningless, despite the arrogant attitude of editors who deem submissions unworthy of publication in their august journal, without bothering to send them to peer-review. A good article will be cited more, no matter where it is published. Articles that aren't useful won't be cited. In other words, articles now can stand on their own, they don't need the support of journals to be useful. The journals, therefore, are middlemen, standing between the producers (the people who do the research and write it up) and consumers (the people who are reading and citing the research). Do we really need journals to arrange peer review? Or is there scope for a journal-agnostic, peer-review service for individual articles?

If individual articles can now stand on their own, how about individual academics? The Khan academy has been described as a revolution in teaching numerous times, and open courses like those offered by MIT have had thousands of students. In many ways universities are middlemen, providing access to resources (academic staff) to consumers (students). Universities provide tuition, consultation (students can ask their instructors for clarification), assessment (tests, assignments and exams), and accreditation (a degree / diploma from an institution has a certain credibility). Tuition can be supplied directly by the lecturer via sites like YouTube. Consultation can be done via discussion boards and live chat. Accreditation remains as an open problem. There are a huge number of accreditations available in a vast range of technical subjects: the IT industry in many ways leads the way in this, with certifications from Microsoft, Cisco, CompTIA and others. Professional organisations like the IEEE publish bodies of knowledge that graduates in certain disciplines are expected to know, and it's only a matter of time before this is expanded to include computational intelligence. Practical work is harder to deal with, but even then the large amount of open source software available means that anyone with a cobbled-together Linux box and a basic internet connection can not only do the lecture and practical work associated with undergrad study but also access the accreditation offered by numerous organisations.

The only problem for which I cannot see an obvious solution is, how would the lecturers get paid? Locking material behind paywalls won't work, people just won't use it. Also, a fixed fee won't work either: $500 might not seem like much for someone in the western world, but for someone in parts of Africa, it's more than they see in a year. The pay-what-you can model might work: this is where someone pays as much as they think something is worth, or as much as they can afford. A few people might take advantage and pay nothing when they could afford to pay, but most people are pretty honest and will pay a fair price. The accreditation agencies could also pay a referral fee to lecturers who direct students to their services, much like the Amazon affiliates program.

Universities would still survive, there still needs to be places where research is carried out, and training of the next generation of researchers (postgraduate students) takes place. The survival of journals is a bit less certain, as self-published peer-reviewed articles are much easier to do. Whatever happens, though, middlemen are on the way out.