Display options
Cite
Zhu J, Fung G, Wong WH, et al. Evaluating the Pros and Cons of Different Peer Review Policies via Simulation. Sci Eng Ethics. 2015;22(4):1073-1094doi: 10.1007/s11948-015-9683-8.
Zhu, J., Fung, G., Wong, W. H., Li, Z., & Xu, C. (2016). Evaluating the Pros and Cons of Different Peer Review Policies via Simulation. Science and engineering ethics, 22(4), 1073-1094. https://doi.org/10.1007/s11948-015-9683-8
Zhu, Jia, et al. "Evaluating the Pros and Cons of Different Peer Review Policies via Simulation." Science and engineering ethics vol. 22,4 (2016): 1073-1094. doi: https://doi.org/10.1007/s11948-015-9683-8
Zhu J, Fung G, Wong WH, Li Z, Xu C. Evaluating the Pros and Cons of Different Peer Review Policies via Simulation. Sci Eng Ethics. 2016 Aug;22(4):1073-1094. doi: 10.1007/s11948-015-9683-8. Epub 2015 Jul 14. PMID: 26169697.
Copy Download .nbib Format: NLM
Share it on

Sci Eng Ethics. 2016 Aug;22(4):1073-1094. doi: 10.1007/s11948-015-9683-8. Epub 2015 Jul 14.

Evaluating the Pros and Cons of Different Peer Review Policies via Simulation.

Science and engineering ethics

Jia Zhu, Gabriel Fung, Wai Hung Wong, Zhixu Li, Chuanhua Xu

Affiliations

  1. School of Computer Science, South China Normal University, Guangzhou, China. [email protected].
  2. Department of Information Technology, Lab Vsio, Hong Kong, China.
  3. School of Decision Sciences, Hang Seng Management College, Hong Kong, China.
  4. School of Computer Science and Technology, Soochow University, Suzhou, China.
  5. School of Computer Science, South China Normal University, Guangzhou, China.

PMID: 26169697 DOI: 10.1007/s11948-015-9683-8

Abstract

In the academic world, peer review is one of the major processes in evaluating a scholars contribution. In this study, we are interested in quantifying the merits of different policies in a peer review process, such as single-blind review, double-blind review, and obtaining authors feedback. Currently, insufficient work has been undertaken to evaluate the benefits of different peer review policies. One of the major reasons for this situation is the inability to conduct any empirical study because data are presently unavailable. In this case, a computer simulation is one of the best ways to conduct a study. We perform a series of simulations to study the effects of different policies on a peer review process. In this study, we focus on the peer review process of a typical computer science conference. Our results point to the crucial role of program chairs in determining the quality and diversity of the articles to be accepted for publication. We demonstrate the importance of discussion among reviewers, suggest circumstances in which the double-blind review policy should be adopted, and question the credibility of the authors feedback mechanism. Finally, we stress that randomness plays an important role in the peer review process, and this role cannot be eliminated. Although our model may not capture every component of a peer review process, it covers some of the most essential elements. Thus, even the simulation results clearly cannot be taken as literal descriptions of an actual peer review process. However, we can at least still use them to identify alternative directions for future study.

Keywords: Peer review; Simulation; Social behavior

References

  1. JAMA. 1994 Jul 13;272(2):91 - PubMed
  2. JAMA. 1998 Jul 15;280(3):240-2 - PubMed
  3. J Clin Epidemiol. 2012 Jan;65(1):47-52 - PubMed
  4. JAMA. 1998 Jul 15;280(3):243-5 - PubMed
  5. Sci Eng Ethics. 2016 Feb;22(1):169-88 - PubMed
  6. JAMA. 1994 Jul 13;272(2):143-6 - PubMed
  7. Nature. 2005 Nov 17;438(7066):282 - PubMed
  8. JAMA. 1998 Jul 15;280(3):246-7 - PubMed
  9. JAMA. 1994 Jul 13;272(2):137-8 - PubMed
  10. JAMA. 1990 Mar 9;263(10):1371-6 - PubMed
  11. Vaccine. 2013 Oct 25;31(45):5147-8 - PubMed

MeSH terms

Publication Types