Open Access Reliability of reviewers' ratings when using public peer review: a case study

 Download
(PDF 355.6 kb)
 

Or sign up for a free trial

Download Article:

Abstract:

If a manuscript meets scientific standards and contributes to the advancement of science, it can be expected that two or more reviewers will agree on its value. Manuscripts are rated reliably when there is a high level of agreement between independent reviewers. This study investigates for the first time whether inter-rater reliability, which is low with the traditional model of closed peer review, is also low with the new system of public peer review or whether higher coefficients can be found for public peer review. To investigate this question we examined the peer-review process practiced by the interactive open access journal Atmospheric Chemistry and Physics (based on 465 manuscripts submitted between 2004 and 2006 receiving 1,058 reviews in total). The results of the study show that inter-rater reliability is low (kappa coefficient) or reasonable (Intraclass Correlation Coefficient) in public peer review.

Document Type: Research Article

DOI: http://dx.doi.org/10.1087/20100207

Publication date: April 1, 2010

More about this publication?
  • Editor in Chief: Pippa Smart
    North American Editor: Judy Luther
    Reviews Editor: Pippa Smart

    Learned Publishing is the journal of the Association of Learned and Professional Society Publishers, published in collaboration with the Society for Scholarly Publishing. The journal is published quarterly in January/April/July/October.

    Learned Publishing articles are available free online to members of ALPSP and SSP.
    ALPSP members: log in to www.alpsp.org. If you do not have a password contact info@alpsp.org
    SSP members: log in to the Member Center using your membership username and password. Further information info@sspnet.org

Related content

Tools

Favourites

Share Content

Access Key

Free Content
Free content
New Content
New content
Open Access Content
Open access content
Subscribed Content
Subscribed content
Free Trial Content
Free trial content
Cookie Policy
X
Cookie Policy
ingentaconnect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more