Anonymous cowards vs the scientific establishment

Battle lines are being drawn on the internet, between the scientific establishment and volunteer vigilantes trying to impose their own vision of the scientific process through “post-publication peer review”.

On one side is the cream of the scientific aristocracy: a professor with a meteoric career trajectory at Imperial College London, one of the best universities in the world, and the top academic publisher, Nature Publishing Group. On the other side: a few anonymous malcontents carping on an obscure web site called PubPeer (welcome to our site!).

A couple of years ago, the professor’s group published in Nature journals a short series of papers reporting an ultra-sensitive assay, in principle for anything that could be recognized by an antibody. The assay, called “Plasmonic ELISA”, was hailed as a breakthrough in diagnostic medicine and is typical of the kind of sensational “high impact” work encouraged by Nature journals. The story was widely reported in the mainstream press.

However, a few scientists felt that the results were a little too good to be true, or at least that unexpected observations were not well explained and that supporting evidence was missing. One of the critics wrote to the journal in private, detailing those concerns. The journal considered them but, with its referees, decided that there was no substance to the complaints. So the publications were not only initially accepted for publication by the journal and its editors, after a typically rigorous review, but their quality was reconfirmed by a second round of review that specifically addressed the criticisms. A stronger proof of quality is hard to come by: Nature says “yes”, twice.

Scientists can be stubborn, especially when they think they are right and have been told they are wrong. The unhappy critic therefore took advantage of our site, which enables anonymous comments on scientific articles, to air his concerns. Site visitors found them convincing and chimed in with their own remarks and analyses. The flow has been essentially one way, with nearly all commenters agreeing that the publications appear to contain serious problems. The difference of opinion with the professor, Nature Nanotechnology and their referees could not be stronger. Only one side can be right. To date, no substantive rebuttal of the criticisms has been posted, though of course the authors and the referees of the papers would be free to defend their work and judgement, anonymously or otherwise. They haven’t responded because dealing with the ignorant internet riff-raff is beneath them or because they have no answer to the criticisms?

Who do you believe – a prestigious professor publishing in a high-impact Nature journal or the anonymous cowards? You can make up your own mind and join in the discussion here (please be polite and factual):
pubpeer.com/publications/23103935
pubpeer.com/publications/3E8208F0654769A44C22D4E78DA2B8
pubpeer.com/publications/3B1B078F8A648D81890AEE2B27C536

8 thoughts on “Anonymous cowards vs the scientific establishment

  1. There is no way anyone can read this and conclude you don’t have an agenda against ‘the establishment’. Why should anyone engage with pubpeer if it is, at best, impartial, and at worst a venue for scientific witchunts?

    • We made this site in order to open review of publications to everyone who can read and interpret them. This is (of course) meant to be disruptive to the current model of releasing scientific data. Isn’t that obvious?

      “Why should anyone engage…?” To make public the discussion of scientific results and not limit those discussions to those directly involved in the act of publishing the data.

      What is a “scientific witch-hunt”? The data are there to be discussed.

      • Nothing wrong with disrupting business or procedural models. But you are taking sides, and that’s doing the community a disservice. Your tone and behaviour on twitter also suggests you’re on a power trip. So again, I cannot see how anybody in their right mind would want to engage in this forum.

        • Please reread the post. We are not taking sides. Those comment threads are one-sided on their own.

          Any “tone” perceived on Twitter has nothing to do with the power we don’t have.

          One wonders what your motivation is to attempt to dissuade people from taking part in open scientific discussions?

          • I’m not attempting to dissuade, I’m just giving you my opinion as a potential user of pubpeer.

            Your post is explicitly setting the scene for a confrontation, and you are insinuating which side you are on. Over the course of the past months, your tone in general has become increasingly confrontational.

            Pubpeer is a great idea. You have the means at your disposal to make this a venue for open scientific discussion, where people feel welcome, or a place for old fiends to come and settle old scores. Right now you are giving the impression of having chosen to go for the latter, and I think that’s a problem for you.

          • I actually agree with Random.

            I haven’t read the threads yet, but the tone of the post, the use of phrases like “cream of the scientific aristocracy”, “riff-raff”, “cowards” – clearly shows which side you take. Or, rather, shows that you are somehow offended that the researchers/journal did not respond to the criticism.

            I agree that this is not a good behavior by the researcher/journal. Criticisms should be addressed and resolved, one way or another.

            But this post clearly shows a bias to the anonymous peers commenting.

  2. One remarkable feature of this story is that, following initial criticisms, the authors and editor accepted that some “details” were missing, but, instead of issuing a correction (retraction?), they published a follow up “protocol” article in the sister journal Nat Protocols with a figure attempting to address the concerns as SI.

    That figure had been made previously specifically to address a concern but was published outside of that context in the Nat Protocol article. At the very least, Pubpeer has enabled to disclose the context of the publication of that figure. As discussed by peers in the threads linked above, that figure solves nothing and seems itself highly implausible.

    Comment for “random”: the post above is not taking side in a scientific argument – which would be inappropriate for the host of those arguments. It is however a sharp criticism of the authors refusal to address a number of detailed scientific reviews, some of which they have already answered in a private correspondence with peer 2 and the editor, i.e. no effort required. Pubpeer goal is to promote open scientific discussion so the refusal of authors to engage, over several weeks, with serious, carefully researched comments, is indeed directly confronting its “agenda”.

  3. Partial reply to “random”. What you are missing is the context of this specific paper (in fact papers, plural) by this group. If you read the comment threads, you will find that all of the comments are basically saying that the results in the paper are internally inconsistent and impossible, with all that that implies. Under the circumstances, I think it is fair to conclude that the “opinion” on/of PubPeer is very unlikely to converge with that of the authors as expressed in the paper. The affair IS polarised and it so happens that on one side you have PubPeer and on the other Nature/Stevens. The blog post accurately describes the situation.

    I suggest that you actually read the papers and the comment threads. The arguments require little more than school maths and some basic biology/chemistry to understand fully. Tell us which side you find more convincing or how you plan to reconcile the arguments of the two sides.

    More generally, your comments suggest that all discussions will be carried out in good faith by both sides. That does happen for a minority of papers discussed on PubPeer, mostly when the authors do have some response/explanation to the criticism. But sometimes the criticism simply highlights a crass fabrication/manipulation, so one side has already demonstrated bad faith. Nearly always such comments attract no defence, understandably.

Leave a Reply

Your email address will not be published. Required fields are marked *