Lie down with pit bulls, wake up with a blogospheric flea in your ear.

This clumsy hatchet job from Nature reporter Declan Butler is beneath him, a poor excuse for journalism and an affront to the respect with which many of his colleagues are regarded by the research community.
Let’s start with the title: “PLoS stays afloat with bulk publishing”. Loaded rhetoric, anyone? The clear implications are that PLoS is floundering (Butler’s own numbers show otherwise!), and that “bulk” is somehow inferior (to, one presumes, “boutique” or some such). PLoS is “following an haute couture model of science publishing” sniffs our correspondant, who goes on to clarify: “relying on bulk, cheap publishing of lower quality papers to subsidize its handful of high-quality flagship journals”.
This emphasis on “quality” and the idea that the same somehow equates with scarcity continues throughout: “the company consciously decided to subsidize its top-tier titles by publishing second-tier community journals with high acceptance rates”, “the flood of articles appearing in PLoS One (sic)”, “difficult to judge the overall quality”, “because of this volume, it’s going to be considered a dumping ground”, “introduces a sub-standard journal to their mix”.
The intent is obvious, and the illogic is boggling. Where does Butler think the majority of science is published? Even if you buy into this nebulous idea of “quality” (one knows it when one sees it, does one not old chap? wot wot?) there can be no “great brand” journals without the denim-clad proletarian masses. All the painstaking, unspectacular groundwork for those big flashy headline-grabbing (and, dare I say it, all too often retracted) Nature front-pagers has got to go somewhere.
It gets much worse, though, when we get some measure of what Butler thinks “quality” means:

Papers submitted to PLoS One (sic) are sent to a member of its editorial board of around 500 researchers, who may opt to review it themselves or send it to their choice of referee. But referees only check for serious methodological flaws, and not the importance of the result.

That, along with an earlier remark about “a system of ‘light’ peer review”, is a blatant and serious misrepresentation of PLoS ONE’s review process. Here’s the actual policy:

The peer review of each article concentrates on objective and technical concerns to determine whether the research has been sufficiently well conceived, well executed, and well described to justify inclusion in the scientific record. […]
Unlike many journals which attempt to use the peer review process to determine whether or not an article reaches the level of ‘importance’ required by a given journal, PLoS ONE uses peer review to determine whether a paper is technically sound and worthy of inclusion in the published scientific record. […]
To be considered for publication in PLoS ONE, any given manuscript must satisfy the following criteria:

  • Content must report on original research (in any scientific discipline).
  • Results reported have not been published elsewhere.
  • Experiments, statistics, and other analyses are performed to a high technical standard.
  • Conclusions are presented in an appropriate fashionand supported by the text.
  • Techniques used have been documented in sufficient detail to allow replication.
  • Reports are presented in an intelligible fashion and written in standard English.
  • Research meets all applicable standards, including the Helsinki Declaration, with regard to the ethics of human and animal experimentation, consent, and research integrity.
  • Report adheres to the relevant community standards for research, reporting, and deposition of data. (Standards PLoS promotes across its journals).

Which is to say that PLoS ONE* holds authors to exactly the same scientific standards that every journal should follow. Which is to say that any methodological flaws, not “only… serious” ones, will see a paper revised, or rejected if the flaws can’t be overcome. Which is to say that PLoS ONE uses peer review to do what it was designed to do, not to create an artificial scarcity from which to milk profit with scant regard for the integrity of the scientific record. That’s not “light” peer review, it’s real peer review.
With this scurrilous parroting of anti-OA FUD, Nature makes pretty clear where its interests and its allies are.  Well, you know what happens when you lie down with pit bulls
There’s a lot more, but that was the issue that pushed my buttons the hardest. See Bora for a roundup of responses; here’s a quick outline of some of the key issues:
Jan Velterop, responding to Butler’s last “investigation” of PLoS finances two years ago, pointed out that it’s ridiculous to expect a new journal with a new business model to break even in a few years, when new journals from established publishers take up to a decade to achieve the same goal; DrugMonkey also mentions the “so what” nature of this complaint. Jonathan Eisen remarks that somehow Butler gets from “PLoS ONE is doing well and making money” to “PLoS is a failure”; go read Jonathan to see how twisted your logic has to be to make that particular trip. (Jonathan also provides an important reminder, that we should not confuse Nature Publishing Group as a whole with their many talented and well intentioned employees!) Grrlscientist observes that, while Butler’s piece makes it sound as though PLoS’ reliance on donations were a bad thing, all journals rely on the donation of time and expertise by unpaid reviewers. Drugmonkey, Jonathan and Grrlscientist all make the point that Nature has its own stable of “second tier” journals with “lower barriers to entry” — the same mechanism for which Butler criticizes PLoS. Stevan Harnad is famous for making the point (here, for example) that if the funds currently draining into subscriptions were used to pay OA costs, there would be an immense improvement in the utility of the scientific record even if there were no financial saving.
Finally, pretty much every commenter has pointed out the glaring lack of any “conflict of interest” statement on the Nature piece — having said which, I’d better make one of my own. It’s well known and obvious at a glance at this blog that my favorite drink is the Open Access Kool-Aid. I have personal friends who work for PLoS, and I’ve previously applied to work there myself.

* originally in lowercase — so much for my snotty (sic)s!

11 thoughts on “Lie down with pit bulls, wake up with a blogospheric flea in your ear.

  1. The circular “quality” spin of Nature and ilk is one of the things that pisses me off too. “We’re kewl because everyone thinks we’re kewl and we brag about being teh kewl.” Well, sack the fuck up. Why not brag for real if you are going to brag? Show us that your hit rate on the most lasting impactful work is better than everyone else’s. Show us that your error catching and methodological strength requirement is better (hah!). Show us what actual fraction of your published work beats those in other journals and what fraction languishes, barely cited. Do some comparisons of critical, equally highly-cited work across many journals within subfields and get a panel of experts to tell us why Nature 200+ cite papers are so much better than any other 200+ cite papers in the literature.

  2. DM: I want to say more about this whole issue, and the related question of “what good are gatekeepers now — isn’t a good search engine enough?”, but not sure I’ll have the time.
    PA: yup, what we are seeing is the desperate flailing of the dying toll-access dinosaur. I don’t think NPG will flail for long — I predict they’ll be the first GlamorMag to go OA.

  3. I don’t think Butler misrepresented the PLoS One peer-review process at all – he emphasized that “importance” (whatever that means) is not a relevant factor at PLoS One. The point is that it shouldn’t be! Butler’s statement just shows that their ‘non-light’ peer-review is the wrong way around: ‘importance’ before scientific accuracy.
    I agree that Nature probably is really scared of OA, but I don’t think going OA will help Nature. I think what would really help scientific publishing is to get rid of 19th century journal technology altogether and have a single repository for peer-reviewed primary literature. Let editors figure out the “importance” once we, the scientists, have vetted the manuscripts.
    Who wants to have ~20,000 scientific journals anyway?

  4. Bjoern, it’s an interesting idea (one repository) and I will have to think more about it before I can say anything coherent, but my first (kneejerk?) response is that if biology has taught me anything it’s that diversity is good.
    It seems inarguable that we want one virtual database — that we want the entire literature to be searchable as one, no matter where it’s stored. But I think there is virtue in having multiple gates, with multiple gatekeepers, through which research reports must pass in order to join that literature.

  5. Right on Bill!
    And I think I agree with you regarding the fact that gatekeepers are going to be obsolete as soon as we find ways to marry search engines with a non-biased way for the crowd to tag and rate papers. Sorta like Faculty of 1000, but open to, you know, everybody.
    Unfortunately, for the vast majority of folks in my lab and friends here, they have never really heard of tagging, folksonomy, or even del.icio.us. So it might take some time. But it’ll happen I think.

  6. Jan, I agree with what you just wrote (and I’m watching keenly to see what Knewco and Wikiprofessional do), but the Anderson piece didn’t do much for me. Too much hype, not enough hard thinking. The biogang on FriendFeed weren’t impressed either (though they were commenting on the Wired article not the Edge.org essay).

  7. I agree that “there is virtue in having multiple gates, with multiple gatekeepers, through which research reports must pass in order to join that literature.”
    However, for a large repository to accommodate the output which is currently published in ~20,000 journals, it needs to have a substantial size, so redress if you feel unfairly treated should not be an issue. If you, as an author have a valid point, there will be always someone who will be able to get it. Even if not, post-publication review will be the ultimate gatekeeper. Because of this ultimate gatekeeper, one rule should be “if there is unresolvable conflict, publish with the conflict stated in the comments to the paper”.
    Exception to this rule would be clearly unscientific rambling, for example.
    And such a system would do away with the endless re-formatting, re-submitting with more/less references, abstracts, word limits and all the other silly 19th century restrictions.
    But again, I agree, the lack of diversity also worries me.
    Perhaps instead of a single repository, one could have a single main gate and then editors decide which paper goes to which journal after peer-review?

  8. Actually, there could be post-publication diversity: Nature, Science, Faculty of 1000, etc. can each choose which papers to highlight. This would get recorded on each paper and they can establish a track record. That way, their statement on being the best authority for the quality of papers can actually be tested objectively.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>