Peer review, the process by which volunteer experts evaluate scientific papers before they are published in professional scientific journals, is a keystone of modern scientific work. According to a study by CSUN Assistant Professor of Biology Nyssa Silbiger and a colleague at Occidental College, peer review can also be an unnecessarily unpleasant experience for many scholars, because reviewers often use the process to lob insults and take cheap shots.
Silbiger and her Occidental College colleague, Amber D. Stubler, were inspired to study the problem of unprofessional peer review after discussing some of the comments they had received on submitted manuscripts. They themselves, and lots of scientists they knew, had had reviewers make comments that were unconstructive, directed at a paper’s authors rather than the quality of their work, based on personal opinions about the authors, or simply mean-spirited. Peer review conventions can potentially provide cover for such behavior, since most scientific journals keep reviewers anonymous, but not so for the authors of a paper under review.
They set up an online survey with a series of structured and open-ended questions about peer review experiences, and recruited other scientists to fill it out. The survey asked for information on whether participants had experienced "unprofessional" peer reviews, qualitative ratings of how those reviews had impacted their confidence and career advancement, and even specific text from unprofessional reviewer reports.
Over 1100 people responded, and more than 600 provided answers to each of the questions about the impacts of unprofessional reviews. A solid majority of participants, 58%, reported having received at least one unprofessional review, and 41% said they had had more than one. Participants provided an eye-watering selection of reviewer comments calling scientific work into question based on the authors’ gender, presumed ethnicity, professional affiliations, and preparedness for a particular kind of research.
Silbiger and Stubler found that the reported frequency of unprofessional peer reviews was essentially the same from participants of different racial backgrounds or differing gender identities. However, they did find substantial differences in the impacts participants reported — women and nonbinary people, and people of color of all gender identities, were more likely to report that unprofessional reviews prompted them to question their scientific aptitude, cost them productivity, or slowed their career advancement. This might reflect the fact that mean-spirited reviews cut deeper for people from groups underrepresented in scientific careers, or the possibility that journal editors take unprofessional comments more seriously when they are directed against members of underrepresented groups — or a combination of both.
Silbiger and Stubler conclude that rude reviews are a potential barrier for increasing the inclusivity of science. They suggest that one proposed remedy, "open review," which entails releasing reviewer identities and even their full comments alongside published papers, could help to combat unprofessionalism and its negative impacts on all scientists:
Although we found clear patterns indicating that unprofessional peer reviewer comments had a stronger negative impact on underrepresented intersectional groups in STEM, all groups had at least some members reporting the highest level of impact in every category. This unprofessional behavior often occurs under the cloak of anonymity and is being perpetuated by the scientific community upon its members. … Our data indicate that open reviews may help curtail unprofessional comments, but more research on this topic is needed.
The paper has been widely discussed, with coverage in the journal Science and Times Higher Education — the full test is available Open Access on the PeerJ journal website