Electronic peer review - possible methods

Electronic publishing gives the opportunity to change the method of peer review to address its inadequacies, particularly its time consuming nature. The xxx e-print archives in Los Alamos were created by Paul Ginsparg as a way for his High Energy Physics community to exchange preprints (unreviewed, unpublished articles) electronically. Now this has become their main method of communication, traditional journals are losing their importance in the field, and the archives are rapidly expanding to incorporate many other scientific disciplines. Being preprints, none of them are reviewed, so perhaps peer review could be dispensed with altogether in future? Harnad (in Matthews, 1995) agrees that the standard at the moment is high, but that this is because "those pre-prints are all destined to appear in rigorously refereed paper journals - it's the invisible hand of anonymous peer review you're seeing in action"

As an attempt to provide a quality control to the xxx archives, Cohen (1997) describes a method of implementing peer review where the author sends the archive number of the article and some information, for example the subject area, to an electronic agent who chooses referees. The referee's criticisms are sent to the (human) editor who decides whether the papers are of a high enough quality to be published. They are then collected into a 'journal' which is really a list of pointers to the papers in the archive, and can then be referred to as 'published' for the sake of the reputation of the author. This method allows the quick publication of results without the delays associated with reviewing and printing, but still retaining a quality assessment. He thinks that it is "desirable" to separate the manipulation of the article (e.g. editing, typesetting) from dealing with the auxiliary information (e.g. indexing, review). The choice of referee by computer is supposed to overcome an editor's possible (subconscious) prejudice towards a small circle of reviewers, by drawing on a large pool, and matching purely on subject knowledge.

Nadasdy (1997) attempts to eliminate the long and laborious process of writing critiques which are then read and assessed by an editor, by using an automatic "interactive vote". Papers published in his electronic journal Electronic Journal of Cognitive and Brain Sciences are linked to an evaluation form with boxes to tick, which the computer uses to calculate a score. If the paper gets over 80% then they are accepted. He admits however that this method relies on the honesty of the reviewer and does not take into account their knowledge of the subject - something that an editor does automatically when picking reviewers. "Just as in any democratic system, voting does not always result in the best decision". Both of these solutions also lose the double blindness of the traditional reviewing system.

Most people agree that a review process is still required in some form, although whether the method will change as a result of new technology is not so clear.

References

Matthews, R. (1995). "Storming the barricades : for years, critics complained that peer review is a way of settling old scores and burying new research. Could the Internet smash the system?", NewScientist, 17.6.95, 38

Exploring the development of the independent, electronic, scholarly journal, by Alison Wells
MSc in Information Management, 1998/1999
Electronic Dissertations Library
© University of Sheffield - Department of Information Sudies (All Rights Reserved)