The current system of scientific publishing has lately received increasing scrutiny and criticism. One essential innovation that has received broad attention in the past decade is open access (OA). OA is now beginning to become a owreality. Hever, in addition to access, a scientific publication system also needs to provide evaluation of papers. Evaluation steers the attention of the scientific community, and thus the very course of science. This essential function is currently served by a secret peer review process administered by the journals. The review process is secret, but its outcome is reflected in the publication of a paper, with the prestige of the journal providing some indication of the quality of the paper. The drawbacks of this form of secret pre-publication peer review are widely acknowledged and include a lack of transparency and reliability of the evaluation process. Critics argue that the current system is a relic of the pre-internet era of scientific publishing. However, it is unclear how a better evaluation system should work. If the evaluation system steers the course of science, it should be designed by scientists – and the cognitive and brain sciences seem well positioned to take on this challenge.

The CBU’s Nikolaus Kriegeskorte and Diana Deca edited a collection of papers in Frontiers in Computational Neuroscience that lay out visions for “open evaluation” (OE), an entirely transparent system for post-publication peer review and rating. The collection is now complete and consists of 18 detailed visions for OE systems. While the papers focus on different aspects of the system, the visions are largely compatible. A synopsis of the 18 visions is given in the Editorial on the collection here.