A lively discussion has developed around some recent high-profile publications reporting some amazing results that were covered in many major media outlets. The comments on one of these papers exemplify perfectly why we created PubPeer and why we feel that post-publication peer review needs to have a formal home. We would like to take a minute to point out these comments and underline why we feel that they are so important.
The traditional routes for scientists to raise questions about a publication are to publish their own paper, write to the authors, or write to the journal. All of these avenues have their disadvantages. A new publication requires a major time (and financial) investment, while an impasse is often reached in interactions with the authors. Both of these approaches identify and expose the critic to reprisals. The final option, writing to the journal, does mostly preserve anonymity, but at the cost of a loss of transparency. Below is a perfect example of this from one of the articles getting a lot of attention on PubPeer at the moment:
In that thread, Peer 2 recounts writing to Nature Nanotechnology, who, to their credit, did take the time to follow up the issue via correspondence with the authors. Ultimately however, although Peer 2 was not satisfied, the editor was, and closed the case. The authors, who have confirmed to us that they are aware of the comments, seem to feel that they have already addressed the issues with the editor of Nature Nanotechnology and that the scientific community should be content with that. However, at the end of the correspondence, the only people aware of the potential issues were Peer 2, the editor of the journal and the authors.
Peer 2 eventually posted the issues on PubPeer, where others interested in the field saw them and added to them with some very thorough reviews of the paper. We are not experts in that field, and we can’t predict what will eventually happen to the paper, but, through reading PubPeer, many others in the field now know about and can evaluate for themselves the issues raised. We consider this to be the main benefit of post-publication review: the dissemination of follow-up discussion so that it is available to all interested parties. Post-publication review can take many avenues (blogs, tweets, etc.) and we encourage the use of all of them. But we also feel that it is essential that others in the field are able to find these reviews easily. That is why a centralized database is essential. If you prefer one of the other methods of review, please also consider cross-posting your remarks to PubPeer, so that your colleagues are more likely to discover your comments.