The problem with academic journals 6

In my previous posts on academic journals (see here, here, here, here, here, and here) I've discussed the major problem with academic journals in the context of the huge cost of accessing the content that the journals receive for free, as well as the importance of open-access journals. This post is concerned with another problem that is becoming apparent with journals: the declining acceptance rate for papers submitted to journals, in attempts to foster an image of exclusivity and quality.

A recent editorial by David Wardle describes a quantitative analysis he performed that compared the acceptance rates of four top-ranked ecological journals with the large open-access journal PLoS One, along with the citation rate of papers published in each. What he found was that the four traditional journals accepted less than 20% of the paper submitted to them, while PLoS One accepted around 69%. However, papers that are published in PLoS One are cited more than papers published in one of the traditional journals. His argument was that the traditional journals rejected papers that were of good scientific quality (that is, they described good work) but were not "worthy" of publication in such "august" journals, with the editors using the excuse that limited page space meant that there wasn't room to print the papers, even though they were quite good. He then goes on to explain that this exclusivity was motivated by a desire to increase the perception of quality of the journals. That is, the editors are trying to foster the impression that the journals must be really good, because they're really picky about which papers they publish.

But, the ultimate measure of the quality of a paper is how often it is cited, as that reflects how useful it is to other scientists, and papers published in the less-exclusive open-access journals are cited more. Thus, the concept that journals with low acceptance rates publish better papers is fatally flawed: these journals are rejecting papers that are scientifically sound and are useful to other scientists.

This leads me to think that the only reason the top journals are the top journals are because people think they are. If someone wants an authoritative citation to back up a statement they make in a paper, they will cite a paper in Nature or Science if they can, because these are the top journals (this doesn't happen much in computational intelligence, because very few papers in this field are published in Nature or Science). But the conclusion of Wardle's study is that acceptance rate is not a reliable metric of the quality of a journal. If anything, it is a measure of the snobbery of a journal.

The purpose of peer review (and of reviewers) is as a crap-filter for papers, to keep work that is incorrectly done or poorly presented from entering the literature. But with exclusive journals, the peer reviewers seem to be spending more time deciding which papers are significant enough to be published in the journal, rather than trying to identify flaws in the work. The whole thing reminds me of the reason the great physicist Richard Feynman quit the US National Academy of Science: because they spent most of their time deciding who was "worthy" of joining the Academy.

Not so long ago, we had to consider the quality of journals because it wasn't feasible to track the impact of a single paper. Now, with tools like Google Scholar, we can track the citation histories of individual papers. In short, the journal in which a paper is published is no longer that important: the usefulness, the contribution of the paper is what is important. By the same token, the quality of an academic is not measured by which institution they work for, but by their contributions. Unfortunately, the bean-counters who make the hiring and promotion decisions, and who make decisions on who gets competitive research funding, haven't grasped this concept yet.

Exclusive journals do not make a good contribution to science, as they keep too much useful material out of the public eye for too long: peer-reviewed open-access journals, with their more liberal acceptance rates, are more important then ever in this situation.

Final reminder: IEEE CIS Facebook Photo Competition

The IEEE Computational Intelligence Society are running a photo competition on Facebook. Go to http://www.watts.net.nz/CIS/contests/photo/2012/ or see the flyer below for further details.

The deadline is three weeks away!

Reminder: paper submission deadline for IJCNN 2013

A reminder that the deadline for submitting papers to the IEEE International Joint Conference on Neural Networks (IJCNN) 2013 is February 1, 2013. This conference will be held in Dallas, Texas, August 4-9, 2013.

Reminder: paper submission deadline for EvoStar 2013

A reminder that the paper submission deadline for EvoStar 2013 is 1 November, 2012. This conference will be held in Vienna, Austria, 3-5 April, 2013.

IEEE Transactions on Autonomous Mental Development: Volume 4, Issue 2, 2012

1. The “Interaction Engine”: A Common Pragmatic Competence Across Linguistic and Nonlinguistic Interactions
Pezzulo, G.
Page(s): 105 - 123
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6006515

2. Interactive Learning in Continuous Multimodal Space: A Bayesian Approach to Action-Based Soft Partitioning and Learning
Firouzi, H.; Ahmadabadi, M.N.; Araabi, B.N.; Amizadeh, S.; Mirian, M.S.; Siegwart, R.
Page(s): 124 - 138
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6032073

3. Tool–Body Assimilation of Humanoid Robot Using a Neurodynamical System
Nishide, S.; Tani, J.; Takahashi, T.; Okuno, H.G.; Ogata, T.
Page(s): 139 - 149
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6095595

4. Are Robots Appropriate for Troublesome and Communicative Tasks in a City Environment?
Hayashi, K.; Shiomi, M.; Kanda, T.; Hagita, N.
Page(s): 150 - 160
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6111246

5. Brain-Like Emergent Spatial Processing
Juyang Weng; Luciw, M.
Page(s): 161 - 185
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6095596

IEEE Transactions on Computational Intelligence and AI in Games: Volume 4, Issue 2, 2012

1. N-Grams and the Last-Good-Reply Policy Applied in General Game Playing
Tak, M.J.W.; Winands, M.H.M.; Bjornsson, Y.
Page(s): 73 - 83
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6203383

2. A Discrete Evolutionary Model for Chess Players' Ratings
Fenner, T.; Levene, M.; Loizou, G.
Page(s): 84 - 93
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6168229

3. Evolving Multimodal Networks for Multitask Games
Schrum, J.; Miikkulainen, R.
Page(s): 94 - 111
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6179519

4. Bitwise-Parallel Reduction for Connection Tests
Browne, C.; Tavener, S.
Page(s): 112 - 119
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6185647

5. Information Set Monte Carlo Tree Search
Cowling, P.I.; Powley, E.J.; Whitehouse, D.
Page(s): 120 - 143
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6203567

6. Benchmarks for Grid-Based Pathfinding
Sturtevant, N.R.
Page(s): 144 - 148
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6194296

IEEE Transactions on Evolutionary Computation: Volume 16, Issue 4, 2012

1. Solving Multicommodity Capacitated Network Design Problems Using Multiobjective Evolutionary Algorithms
Kleeman, M. P.; Seibert, B. A.; Lamont, G. B.; Hopkinson, K. M.; Graham, S. R.
Page(s): 449 - 471
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151105


2. An Integrated Neuroevolutionary Approach to Reactive Control and High-Level Strategy
Kohl, N.; Miikkulainen, R.
Page(s): 472 - 488
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151106


3. A Process Algebra Genetic Algorithm
Karaman, S.; Shima, T.; Frazzoli, E.
Page(s): 489 - 503
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6045330


4. Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multiobjective Optimization
Schutze, O.; Esquivel, X.; Lara, A.; Coello, C. A. C.
Page(s): 504 - 522
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151115


5. Promoting Creative Design in Interactive Evolutionary Computation
Kowaliw, T.; Dorin, A.; McCormack, J.
Page(s): 523 - 536
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151108


6. Effects of Iterated Interactions in Multiplayer Spatial Evolutionary Games
Chiong, R.; Kirley, M.
Page(s): 537 - 555
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151098


7. A General Framework of Multipopulation Methods With Clustering in Undetectable Dynamic Environments
Li, C.; Yang, S.
Page(s): 556 - 577
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151109


8. On the Design of Constraint Covariance Matrix Self-Adaptation Evolution Strategies Including a Cardinality Constraint
Beyer, H.-G.; Finck, S.
Page(s): 578 - 596
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6151095

IEEE Transactions on Fuzzy Systems: Volume 20, Issue 4, 2012

1. Finite-Time $H_{infty}$ Fuzzy Control of Nonlinear Jump Systems With Time Delays Via Dynamic Observer-Based State Feedback
He, S.; Liu, F.
Page(s): 605 - 614
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094200

2. A Practical Approach to R&D Portfolio Selection Using the Fuzzy Pay-Off Method
Hassanzadeh, F.; Collan, M.; Modarres, M.
Page(s): 615 - 622
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6109284

3. Fuzzy Hardware: A Retrospective and Analysis
Zavala, A. H.; Nieto, O. C.
Page(s): 623 - 635
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6111466

4. On Robust Fuzzy Rough Set Models
Hu, Q.; Zhang, L.; An, S.; Zhang, D.; Yu, D.
Page(s): 636 - 651
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6111464

5. Fault-Tolerant Control for T–S Fuzzy Systems With Application to Near-Space Hypersonic Vehicle With Actuator Faults
Shen, Q.; Jiang, B.; Cocquempot, V.
Page(s): 652 - 665
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6111465

6. Constrained Fuzzy Hierarchical Analysis for Portfolio Selection Under Higher Moments
Nguyen, T. T.; Gordon-Brown, L.
Page(s): 666 - 682
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6112209

7. An Integrated Mechanism for Feature Selection and Fuzzy Rule Extraction for Classification
Chen, Y-.C.; Pal, N. R.; Chung, I-.F.
Page(s): 683 - 698
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6112676

8. Generalizing the Decentralized Control of Fuzzy Discrete Event Systems
Jayasiri, A.; Mann, G. K. I.; Gosine, R. G.
Page(s): 699 - 714
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6112712

9. Participatory Learning of Propositional Knowledge
Yager, R. R.
Page(s): 715 - 727
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6119214

10. The $K$-Means-Type Algorithms Versus Imbalanced Data Distributions
Liang, J.; Bai, L.; Dang, C.; Cao, F.
Page(s): 728 - 745
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6121900

11. Stress Monitoring Based on Stochastic Fuzzy Analysis of Heartbeat Intervals
Kumar, M.; Neubert, S.; Behrendt, S.; Rieger, A.; Weippert, M.; Stoll, N.; Thurow, K.; Stoll, R.
Page(s): 746 - 759
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6127913

12. Fuzzy Preferences in the Graph Model for Conflict Resolution
Bashar, M. A.; Kilgour, D. M.; Hipel, K. W.
Page(s): 760 - 770
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6127912

13. Observer-Based Adaptive Fuzzy Backstepping Output Feedback Control of Uncertain MIMO Pure-Feedback Nonlinear Systems
Tong, S. C.; Li, Y. M.; Shi, P.
Page(s): 771 - 785
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6126023

14. On the Use of a Fuzzy Object-Relational Database for Flexible Retrieval of Medical Images
Medina, J. M.; Jaime-Castillo, S.; Barranco, C. D.; Campana, J. R.
Page(s): 786 - 803
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6208854

Reminder: paper submission deadline for AROB 2013

A reminder that the deadline for submitting papers to the 18th International Symposium on Artificial Life and Robotics (AROB) 2013 is 1 September, 2012. This symposium will be held in Daejeon, Korea, January 30 - February 1st, 2013.

IEEE Transactions on Neural Networks and Learning Systems; Volume 23, Issue 8, August 2012

1. Title: Twenty Years of Mixture of Experts
Authors: Seniha Esen Yuksel; Joseph N. Wilson; Paul D. Gader
Page(s): 1177 - 1193
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6215056

2. Title: Constrained Empirical Risk Minimization Framework for Distance Metric Learning
Authors: Wei Bian; Dacheng Tao
Page(s): 1194 - 1205
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6203595

3. Title: Scale-Invariant Amplitude Spectrum Modulation for Visual Saliency Detection
Authors: Dongyue Chen; Hao Chu
Page(s): 1206 - 1214
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6212362

4. Title: Relaxed Fault-Tolerant Hardware Implementation of Neural Networks in the Presence of Multiple Transient Errors
Authors: Hamid Reza Mahdiani; Sied Mehdi Fakhraie; Caro Lucas
Page(s): 1215 - 1228
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6213557

5. Title: Mapping Dynamic Bayesian Networks to $alpha$-Shapes: Application to Human Faces Identification Across Ages
Authors: Djamel Bouchaffra
Page(s): 1229 - 1241
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6215055

6. Title: Predictive Approach for User Long-Term Needs in Content-Based Image Suggestion
Authors: Sabri Boutemedjet; Djemel Ziou
Page(s): 1242 - 1253
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6218198

7. Title: SOMKE: Kernel Density Estimation Over Data Streams by Sequences of Self-Organizing Maps
Authors: Yuan Cao; Haibo He; Hong Man
Page(s): 1254 - 1268
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6218200

8. Title: Reinforced Two-Step-Ahead Weight Adjustment Technique for Online Training of Recurrent Neural Networks
Authors: Li-Chiu Chang; Pin-An Chen; Fi-John Chang
Page(s): 1269 - 1278
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6218199

9. Title: Spatial Gaussian Process Regression With Mobile Sensor Networks
Authors: Dongbing Gu; Huosheng Hu
Page(s): 1279 - 1290
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6218781

10. Title: Adaptive Data Embedding Framework for Multiclass Classification
Authors: Tingting Mu; Jianmin Jiang; Yan Wang; John Y. Goulermas
Page(s): 1291 - 1303
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6220302

11. Title: Study on the Impact of Partition-Induced Dataset Shift on $k$-Fold Cross-Validation
Authors: Jose García Moreno-Torres; José A. Sáez; Francisco Herrera
Page(s): 1304 - 1312
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6226477

12. Title: Kernel Recursive Least-Squares Tracker for Time-Varying Regression
Authors: Steven Van Vaerenbergh; Miguel Lázaro-Gredilla; Ignacio Santamaría
Page(s): 1313 - 1326
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6227361

13. Title: Discrete-Time Neural Inverse Optimal Control for Nonlinear Systems via Passivation
Authors: Fernando Ornelas-Tellez; Edgar N. Sanchez; Alexander G. Loukianov
Page(s): 1327 - 1339
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6238379

14. Title: Equilibria of Perceptrons for Simple Contingency Problems
Authors: Michael R. W. Dawson; Brian Dupuis
Page(s): 1340- 1344
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6213123

Fraud in science

Ars Technica has a slightly tongue-in-cheek article on how to commit scientific fraud and get away with it. The article discusses eight points:

1. Fake data nobody ever expects to see
2. Work with many collaborators
3. Tell people what they already know
4. Don't do research anyone cares about
5. Don't publish in journals focused on your field
6. Distribute responsibility
7. Don't plagiarize
8. Don't duplicate images

It seems to me that, unfortunately, computational intelligence (CI) is more susceptible to many of these methods than many other fields. I sometimes joke that I am fortunate to work in a field where I can perform solid research by making stuff up as I go along, by which I mean that developing algorithms or techniques is often a more creative process than, for example, research in biology or physics. But think about how many CI papers you've seen that don't make the data available (point 1), or even describe its statistical parameters?

A long list of co-authors is not as common in CI (point 2) as it is in other fields, but I have seen many, many papers that are going over the same topic as has been covered many times before (point 3). Also, many, many papers cover minimal, slightly incremental "improvements" to existing algorithms that are of little true interest to most other researchers (point 4).

While one of the great joys of working in computational intelligence lies in the broad range of applications the field can be applied to, it does provide more opportunity to publish in journals that specialise is other fields (point 5).


The remaining three points (6-8) are more concerned with how not to get caught, or rather, how not to draw attention to yourself while committing fraud.

Fraud is always a problem, and I don't think that it is any less common in CI than in any other field. A greater emphasis on the use of statistics in CI papers would help guard against fraud (see my posts here and here about increasing the statistical basis of CI papers). But apart from that, we still depend on the honesty and integrity of the authors.

Reminder: IEEE CIS Facebook Photo Competition

The IEEE Computational Intelligence Society are running a photo competition on Facebook. See the flyer below to find out how to enter.

More on open access journals

Continuing my series of posts on open access journals (see here and here), this article by Simon Owens in U.S. News is an excellent and detailed review of the debate. The article compares open access journals to e-books: while e-books have existed for a long time, it is only in the last five years that they have really taken off, after reaching a tipping point. Owens argues that open access journals have reached that tipping point, and the academic journal publishing business (known for the huge profits they extract from university libraries) is on the verge of serious disruption.

I tend to agree with his assessment, open access journals have been flying largely under the radar for a long time, but I get the sense that they are becoming more accepted among the top researchers: when more top researchers publish in open-access journals, they will gain credibility.

The old publishing model is being destroyed by greed: journals are just too expensive, and suck too much money out of universities that should be spent funding research and paying people's salaries. Open access is the future of scientific publishing.