Publishing a paper is still considered a definitive event. And what could be more definitive than publishing two Nature papers back to back on the same subject? Clearly a great step forward must have occurred. Just such a seismic event happened on the 29th of January, when Haruko Obokata and colleagues described a revolutionarily simple technique for producing pluripotent cells. A short dunk in the acid bath or brief exposure to any one of a number of stressors sufficed to produce STAP (Stimulus-Triggered Acqusition of Pluripotency) cells, offering enormous simplification in stem cell research and opening new therapeutic avenues.
Nature were presumably conscious of a potential problem when processing the paper. The field of stem cell research had seen the huge Woo-Suk Hwang scandal. It was also shaken by the hasty three-day review by Cell of a paper from the Mitalipov group that contained several very careless errors (and whatever happened to those promised external verifications of its central claims?). Obviously, it wouldn’t do to get caught out like Science and Cell.
So Nature and its referees must have known that these papers would have a bullseye painted on their backs. And, knowing this, one suspects that they would have been extra-careful. Their most trusted referees would take their time to ensure that the authors had dotted all the ‘i’s and crossed all the ‘t’s. The papers indeed underwent a respectable 9 months gestation from submission to acceptance; the authors even complained about the strictness of the referees. So how would Nature’s high(est) quality output get on in the big bad world? One would maybe have expected a few quibbles about finer points of interpretation, but surely the critics of pre-publication review would find little ammunition in such carefully prepared papers.
The papers came out to blaring publicity. The mainstream media were all over the story. Nature had another breakthrough; Riken and Harvard were revelling in their glory. But the internet allows quite accelerated feedback, from anybody and everybody. Inveterate stem-cell blogger Paul Knoepfler was immediately on the case, sharing his field’s bewilderment, a mixture of skepticism and hope. Our site received some chatter, mostly about the predicted consequences of the stated T-cell origin of the stem cells.
Then, on February 4th, less than a week after publication, an anonymous comment on PubPeer pointed out that a gel showed signs of having a lane spliced in (http://imgur.com/1nBfKTr). Although some people don’t consider this practice very serious, most ‘Peers’ take quite a dim view of it. In any case, an unannounced splice was potentially deceptive and probably caused people to examine the papers with a more critical eye. Over the following weeks, quite a number of comments highlighting small (inconsistent scale bars) and potentially serious (possible figure duplications) problems were posted. Numerous comments linked back to Japanese blogs, whose interest was understandable. At the very least, the papers were shown to contain a disconcerting number of unfortunate errors.
In parallel, Knoepfler was giving the stem cell field weather report (turning gloomy), interviewed coauthors Charles Vacanti and Teruhiko Wakayama, and also ran an innovative crowd-sourced replication page, which soon showed that promises of simplicity and hopes of generality would not easily transfer from the abstract to the lab. We learnt that both Riken and Nature were ‘investigating’. Questions were also raised about papers by Vacanti and Wakayama.
Suspense turned to outright febrility following new comments posted on PubPeer this weekend, which may turn out to be the straws that broke the donkey’s back. They claim that figure panels in one of the Nature papers appear to have been duplicated from different experiments in Obokata’s PhD thesis (unfortunately not online). Knoepfler then ran a hugely entertaining piece speculating that Riken might choose to precipitate a retraction in order to avoid being blamed by Harvard, presumably according to the theory that senior authors are blameless victims, especially if they work at Harvard. We also heard that coauthor Wakayama now says he has lost faith in the work, despite being recently quoted as having replicated Obokata’s technique for himself.
One suspects that the denouement is not far off. But some conclusions can already be drawn. We see that post-publication peer review easily outperformed even the most careful reviewing in the best journal. The papers’ comment threads on PubPeer have attracted some 40000 viewers. It’s hardly suprising they caught issues that three overworked referees and a couple of editors did not.
Science is now able to self-correct instantly. Post-publication peer review is here to stay.