A Heretic in Charleston

I was trying to figure out a theme to use for my comments at the panel discussion in Charleston on Friday.  Anthony Watkinson had done a superb job of framing a series of questions for us to respond to during the session "Open Access - Beyond Declarations".   But with only ten minutes of speaking time available, it wasn't going to be possible to address each of them as thoroughly as they deserved.  We'd have to pick and choose.

Among other things, Anthony asked, Is the achievement of Open Access to (all) scholarly communication a moral imperative, or is it one where advantages and disadvantages have to be weighed and evidence adduced? 

The open access moralism on the part of some of the partisans has been extremely damaging to the entire discussion, so I ended up characterizing myself as perhaps an Open Access Heretic, pointing out that, "Martin Luther continued to believe in Jesus.  He just quit believing in the Pope."  That's a fair metaphor for the evolution of my views.

When one takes the strong moralistic approach, the open access all or nothing approach, and treats it as if it is the most important issue in scholarly publishing, then one is essentially absolved from the difficult consideration of social costs.  If one feels that the social benefits of open access are clearly and completely overwhelming, then one is compelled to push for whatever solutions might point in that direction and let the chips fall where they may.   But to righteously ignore the fact that some of those chips may fall very heavily indeed is irresponsible.

The result of this has been to create a climate of strong distrust among the various stakeholders and to lead both sides to make tenuous and twisted arguments that may make good soundbites, but don't really hold up under close analysis.  In the battle of the press releases, all sides present overreaching arguments, designed, finally, to score points, rather than to push the discussion forward seriously. 

I tried to illustrate this with the "taxpayer rights" argument.  In a mom and apple pie kind of way, the statement that taxpayers should have immediate access to the results of federally funded research is trivially true.  But this could easily be met by having scientists write up the results of their work and post it to publicly available websites.   This, however, is clearly not what those who are making the argument would be satisfied with -- they still want the benefits of the peer review and editing processes that are part of the publication system and that are not, under the traditional system, paid for by the taxpayers.   It is the subscription system that currently pays for those added benefits.  So in order to include them, we need to shift the funding streams so that the taxpayers are paying for those benefits as well.  Indeed, this is what the move towards having funders pay for publication costs is all about, and that approach seems perfectly reasonable and logical to me.  It is not, however, without social costs, and the blithe response on the part of the advocates, who dismiss the concern about costs by saying it is such a tiny portion, maybe 2% or so, of overall NIH funding, is simply not sufficient.  At a time when the NIH budget is flattening and competition for grants is becoming tighter and tighter (at present, NIH is funding just under 20% of approved applications), and promising young scientists are leaving academic careers because they're not able to get that all important first grant, shifting even 2% of the budget toward publication is not a trivial matter.  Open access advocates need to do a much better job of making a compelling and detailed case for why the benefit is worth the cost.

The taxpayer rights argument is the soundbite hook on which FRPAA hangs as well, and it is a soundbite that plays well with members of congress and in the press.  But, of course, FRPAA itself is a compromise and doesn't provide any more immediate access than the Highwire publishers do independently.  "Libraries aren't going to cancel subscriptions if there is an embargo," say the partisans.  Since this seems so obvious to them, they accuse the publishers who are opposed to FRPAA of bad faith for claiming that they are concerned about the survivability of their organizations.   But since the people who are claiming that FRPAA doesn't threaten subscriptions are the same people who are on record as taking the moralistic view that everything ought to be immediately open, can one really blame the publishers for being convinced that FRPAA is just one step along the way, and that if they give in to this "compromise" it will simply encourage the partisans to continue to push for complete and immediate access?

Open access moralism has poisoned the debate, generated tremendous distrust, pushed people (most of whom I believe are essentially well meaning) into making tendentious and unsupportable arguments (on both sides), and made it far more difficult to build the kinds of alliances that might actually enable us to develop a social benefit calculus that could lead to positive changes that don't carry the burden of unintended negative consequences.

By the time I got to the speakers' platform, I was fairly convinced that I was going to seriously annoy everyone in the room.   So I was somewhat surprised, and deeply gratified, by the number of positive comments I got from people immediately afterwards and throughout the rest of the day.  Perhaps there are indeed an increasing number of people who are trying to find a useful middle ground.   (Which is not to say that I didn't annoy plenty of members of the audience -- I'd've felt a failure if that had been the case).

I hope that we've reached a point where we can do some bridge building among the various stakeholders and do the hard work of seriously analyzing the social costs & benefits of various open or enhanced access approaches.   A much stronger and richer scholarly communication system is within reach, with greatly expanded opportunities for access and synthesis, but it will require careful work among all of the stakeholders to bring it to fruition.  The battle of the press releases isn't going to get us there.

Comments

[Marcus]

I always knew you were a heretic, Scott, ever since you started wearing that big black hat.

Open access is one of many cases in which groupthink crowds out creative think. So heretics are essential if we're actually going to get anywhere.

With that said, I must confess: On many occasions, I've argued some version of, "Publication costs are just a tiny fraction of the overall grant. Why not roll those costs into the grant award and get on with open access? Let freedom ring!"

So, OK, even if publication costs are only 2% of overall costs, that's still real money. The question for open access advocates then becomes, is there a principled justification for asking taxpayers to foot the bill for quality publication? If we have to pay dearly for it, is it still worth having?

I say yes. Publication is part and parcel of the research process, not an entirely new step that suddenly comes along at the end of it. From the beginning, researchers are thinking about potential papers.

But quality editing costs money. Let's imagine two scenarios, in which the NIH funding levels are exactly the same:

A. Traditional model--100 papers published in a given year, 80% of which are open access in some fashion within six months to a year.

B. Open access model--80 papers published in a given year. The same amount of money goes less far in this scenario, because publishers have to recoup their costs up front. But we have 100% immediate open access to all articles.

Obviously this a contrived scenario. But if you can bear with that limitaion, I argue that scenario B is better. *All* research deemed worthy of publication by NIH is immediately available for other scholars.

Is Scenario A awful? Are the publishers who prosper under Scenario A venal human beings? No on both counts. But without vitriol, we can argue that there is a public policy imperative in favor of Scenario B.

[T Scott]

But you're not arguing that there is a public policy imperative -- you're simply asserting it. If the actual cost is that some research doesn't happen at all because it doesn't get funded, and some promising careers are abandoned altogether because of the difficulty of getting that first grant, what is the calculation that you use to prove that a modest increased accessibility to some papers is worth that cost?

[Marcus]

Most evidence now shows a strong citation advantage of open access articles. See http://biology.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pbio.0040157 for a recent example. An earlier example in the library literature is: http://www.la-press.com/include/Antelman.pdf

So, under full OA, new science builds faster on the science that already exists. This is the philosophical argument. If there's a casualty to some promising careers in the transition to full OA, that's obviously terrible for the scientists and for society. But it is not a strong enough reason to maintain the current publishing model, all things considered.

[MarkD]

It has been my experience that librarians grossly under-estimate the complexity and cost of peer review and editing.

Two percent is an absurd estimation, and as far as I can tell, completely baseless. It is more likely that these costs would be closer to 20% then to 2%. The problem is, I haven’t seen any hard data that documents the cost of peer review, redaction, and publishing. Everyone throws numbers around as if they were confetti. We are all, supposedly (publishers and librarians) in the scientific/technical community, yet so very few people take a scientific approach to this issue.

The first step on the road to open access, should be a review of the processes and costs associated with scientific publication. Sounds like a good paper for the library association journal. Any librarians out there that want to tackle this paper?

And as for the publishers, if they really do wish a dialogue, then why don't they reveal their redaction costs? Any takers out there in the publishing world?

[Marcus]

We're all assuming that the current peer review model is worth preserving, as it is. While that model has served science well for centuries, it too is in great flux. Today's physicists routinely post their work without peer review, and *then* their colleagues separate the wheat from the chaff.

Yes, librarians do underestimate the costs of peer review and other editorial functions. But we shouldn't assume that peer review will not continue to evolve, hopefully growing less expensive as it becomes more informal.

[MarkD]

Marcus, there is a difference between medicine and physics. If physics papers are inaccurate the consequence is nothing more than theoretical. If a medical paper is inaccurate; then people die. The stakes are much higher in medicine.

If you want a model for efficient peer review, then perhaps you should look to Elsevier. i've heard librarians often complain about Elsevier’s ‘obscene’ profits. It has been my contention that Elsevier’s profits are more then a function of its prices. Elsevier profits also reflect its efficiency. I suspect that Elsevier is so profitable because it is more efficient in redaction and production processes then societies and associations. Increased efficiency does not necessarily mean lower cost for the user. It could merely result in higher profits for the producer.

[Bill Hooker]

"If physics papers are inaccurate..."

Space shuttles fail to launch and bridges collapse. While such things do happen, patients also die.

Clinicians don't browse the research literature and suddenly decide to try this kinase inhibitor or that surgical procedure, they work on clinical trials or await the results of same and only test on patients those models for which they have good empirical evidence. Whatever its form, there will always be a layer of peer review and then a second layer of community consensus between the lab bench and the patient bed. I think it would be difficult to make the case that arXiv is less efficient at carrying out peer review and facilitating the development of consensus models than toll-access journals.

[Marcus]

Good point Mark--The physics arXiv example is overworked.

But some physicians must be interested in the concept of collaborative, open, dynamic peer review. Take a look at http://www.plosone.org/

But let's say PLos One fails, and that standard peer review is the only model that works going forward, forever. I still argue that taxpayer subsidies to cover peer review, in order to maximize dissemination of taxpayer funded work (which is certainly not all research), are appropriate whatever the cost.

For publicly funded research, the public good argument for open access is two-fold:

1. All taxpayers have a right to read what they've paid for.

2. Openly accessible articles are more readily cited by other scholars, thereby hastening the development of science. As President Bush might put it, this is more bang for your buck.

[MarkD]

Sorry Bill, yes indeed clinicians will read a paper and suddenly change their prescribing habits. We just conducted a study here in Australia, 71% of the physician respondents said they were influenced by reprints of articles (usually provided by the pharma companies) and that they were prepared to change their prescribing based on those papers.

Doctors will indeed read and change habits, based on these papers, and that was my point. The editing process (after the paper is written and reviewed) is critically important. Mistakes are commonly found - even in peer revieweed papers. People, including doctors, can be lazy in following up on the facts.

Yes, Marcus I agree, taxpayers have a right to read what they have paid for. But do US taxpayers have a right to see what the Dutch taxpayers have paid for? What about the fact that only a small minority of medical research is funded by US taxpayers? A large precentage of the important advances are funded privately. These studies are no less important in advancing medicine? Should the public be denied access to these papers; merely because they were funded privately?

[Bill Hooker]

Mark: huh. I guess I don't talk to enough clinicians. The part about "usually provided by the pharma companies" is worrisome. Worse, altering prescribing habits on the basis of individual articles seems like very bad practice to me. It removes the "consensus" layer of checks-and-balances that is largely the reason science produces reliable knowledge. (That's not to say that it isn't a common -- 71%! -- practice.) As you point out, such practice raises the stakes on catching errors in every individual article.

It then becomes important to point out that OA is entirely compatible with the standard, currently-operative peer review methodology and infrastructure. Widespread adoption of OA need not, and indeed should not, mean radical or rapid alteration of peer review. It might make it easier for unscrupulous pharma companies to pull up unrefereed preprints with which to influence clinicians, but I'm not convinced that that is an argument against OA.

Regarding efficiency: I think it's pretty well established that Elsevier's profit margins are due to increased charges, not increased efficiency. (See various links at http://www.econ.ucsb.edu/%7Etedb/Journals/jpricing.html, for example.) Moreover, OA journals, by removing the need for gatekeeping (subscription management, authentication, and so on) can enjoy a substantial increase in efficiency over toll-access journals.

This last is also part of Peter Suber's reply to your point regarding worldwide vs national access; see http://www.earlham.edu/%7Epeters/fos/newsletter/09-04-03.htm#taxpayer and scroll down to objection #4.

Finally, you claim that "only a small minority of medical research is funded by US taxpayers [...] A large precentage of the important advances are funded privately". Could you point me to your sources on that? It's an important question, worldwide and in the US: how much research is privately funded, and how much is paid for by tax money? I'd be grateful for any information you can point me to. (In re: denying the public access, private funders are of course at liberty to do that in a (relatively) free market system. It's up to OA advocates to persuade them that it's not in their best interests!)

[KarenChr]

I'm a small publisher doing reference and trade/professional titles, working in new areas with hundreds of academics. No surprise that people are suggesting we start publishing a journal or two or three. How can we make this pay, do it right (economically and ethically), and do something that uses the best of open source and social media? I'll be following this discussion in the hope of figuring out how to proceed. Karen Christensen, karen [at] berkshirepublishing [dot] com.

[MarkD]

Bill, I don’t claim that this is an exhaustive analysis. But I am the source. I looked at a leading international medical journal over a period of a few months. Most medical journals list the source of the funding for each article published.

I came up with these stats

NIH funded research 23%
Various European Governments funded research 12%
Private institutions 7%
Pharmaceutical companies 35%
University 14%

The rest was a combination of the above.

My analysis is too narrow I fear. I haven’t had time to follow up and do a more extensive analysis. But again, it sounds like a great idea for a paper in the JMLA. One could look over several years in say the top 10 medical journals and tally it all up. I am the first to admit that my data is not nearly extensive enough. But as far as I can tell, no one else has even tried – even though the data is readily available. Bill, please don’t post these stats elsewhere, this was something I did over a couple of hours one weekend and I would hardly call it a scientific approach. Though I think it a worthy project for an ambitious young librarian, don’t you? I am the first to admit that my numbers could be unrepresentative. I think the methodology is sound though if it were expanded to the top 10 – 20 journals for the period of an entire year. If you stick to the big international titles you should get a very representative sense of what is happening in the market as a whole.

Another potential source of information, annual reports of pharma companies – all would list the amount of money they spend in research each year. Quickly reviewing annual reports I get these numbers for private expenditure on research and development (2005):

Pfizer $7.45bn
Glaxo $3.33bn
Merck $3.85bn

As you can see, these three companies alone spend far in access of the NIH- so it comes as no surprise that more papers are generated by private concerns.

Karen, that is the question of the hour. To-date, no one has been able to answer that question. Maybe more time is needed to see ‘what works’ and what doesn’t. My concern is the ethics of open access. I think that it is a system potentially open to abuse. But I keep thinking about it. I keep trying to work my head around the ethics and the business model. How do you keep it clean? How do you make it pay for itself? Is government funding the answer? Hopefully they would do a betteer job with funding research then they have in running Iraq.

[Bill Hooker]

Karen, if you don't already know about it, I refer you to Peter Suber's blog, Open Access News (http://www.earlham.edu/~peters/fos/fosblog.html). The blog will keep you up-do-date with OA, and the sidebar to the right contains a bunch of useful links to background information. There's also my 3QD essay (http://3quarksdaily.blogs.com/3quarksdaily/2006/10/the_future_of_s_1.html), which may contain a few additional links of interest to you.

Mark, thanks for the numbers! (I won't spread them around -- we are just poking at the problem here, trying to get a sense of how a solution might work.) I like the idea of scanning the "funded by" attributions in various major journals, but it is limited by the fact that no numbers are mentioned. A project that lists a grant from the NIH and support from, say, Glaxo, does not necessarily say how much each funder contributed. I think you are quite right though, the basic idea would provide useful information if expanded to cover a few dozen major titles over, say, the last year.

In re: annual reports, the NIH budget is closer to $30 billion (http://www.nih.gov/about/budget.htm). Economist Dean Baker has some interesting numbers on the pharmaceutical industry (http://maxspeak.org/mt/archives/001704.html), which I take to be the largest source of private investment in biomed research.

[Marcus]

What a great discussion! After a few days delay, I log on this morning and see many interesting new comments. But I'd like to step back a bit and respond to Mark's comment aout whether US taxpayers have the right to resources subsidized by Dutch taxpayers. Elsevier, of course, is a Dutch company. So the standard line that "US taxpayers have a right to see what they've paid for" becomes much more complicated in a global publishing marketplace.

This is why I think that the proposed Federal Research Public Access Act (FRPAA), proposed by Senators Cornyn and Lieberman, is so important. It would force the issue, and untangle the knots that Mark has described.

FRPAA would require that US publicly funded research be available in some fashion within 6 months of publication. If Elsevier refuses to publish under these conditions, scholars would go elsewhere in order to comply with the law. Maybe there would be international summits in order to harmonize publishing laws, since this is a global market.

We won't know until the law is passed. My prediction is that savvy publishers will evolve to accomodate these new realities. This would be a "natural experiment," which I think is the only way to resolve this debate.

[MarkD]

I used the Netherlands as an example for two reasons; 1) it is the home of Elseveir and Kluwer; 2) it punches above its weight in medical research. Dutch government funded institutions seem to produce a large number of medical papers. Bill, my point regarding pharma research: just three companies R&D spend nearly equals the total spent by the NIH. There are more then 30 large pharma companies, they have to be spending far in access of the NIH. This does not mean that they are spending their money wisely, however.

Marcus, you hit on my point precisely. The OA movement is too US centric, Americans always think they are the centre of the universe. Much happens outside the US. That is the problem with the Cornyn/Lieberman bill – it addresses a small minority of research. It would only have a true impact if it served as an example. And then there is the issue of jointly funded research. A large segment of clinical trials are multi-national, multi-institutional, public/private collaborative efforts. Where do these studies fit into the mix? Would Cornyn/Lieberman force jointly funded papers into the six month regime? Where is the cut off? What impact will the legislation have on collaborative efforts?

Bill, you have stumbled on to my main concern about open access. Everyone on this blog is in the academic community – so you focus on the issues that are important to that community. However, you represent only a very small group of people who use these research results. Medical research means money, the results of a study can mean literally billions of dollars to investors. There is indeed a risk of manipulation of the data. One of the greatest services the current peer review process provides is a check on those private financial concerns. There are responsible publishers out there that take their policing responsibility seriously. The weapon they use to protect the truth is copyright law. One of the questions the open access movement must address is this; how do you insure (in an open access regime) that the information is not manipulated and altered to serve a private end. When billions of dollars are involved, I can assure you this provides a strong incentive to influence how data is presented. The current peer review copyright regime is not perfect, but it does at least provide a check. I see no inherent check in open access. If you are serious about open access then you must address this issue.

Bill regarding phamra distribution of reprints. Yes it does pose a risk. On the other hand, for a large segment of the medical community, the pharma companies are the most important source of information. If the industry did not hand out peer reviewed reprints from prestigious journals large segments of the community would never read the literature. Yes I have data to back that statement up (but it is proprietary and I do not have permission to disseminate it).

[Bill Hooker]

Mark, of the $41 billion the pharmaceutical industry as a whole spent on R&D in the US in 2004, $27 billion was spent on copycat drugs -- that is, research that solved no problem, that was aimed solely at exploiting intellectual property laws for profit. The remaining $14 billion is approximately half the NIH budget. (Sources in the links in my last post, but some are "russian doll" -- that is, the linked post leads to another paper, which references another, and so on.)

More importantly, what OA model are you looking at, that has no peer review? How are papers in any of the 2500-odd OA journals that are currently operating any more likely to be fraudulent than papers in a toll-access journal?

[Bill Hooker]

Oops, my mistake. There are, of course, plenty of unreviewed OA digital resources, including biomed. The DOAJ (http://www.doaj.org/) is my first stop for OA journals, but it's not the only list out there! The DOAJ lists about 2500, all peer reviewed. There are larger lists that include non-peer-reviewed OA journals, and services like OAIster presumably do not discriminate between preprints and postprints, nor between refereed postprints and unrefereed ones.

My usual response to "bad actor" objections is to point out that they are everywhere, and it is a mistake to allow them to dictate terms. It's also true that the unrefereed articles are already available for dissemination by the unscrupulous, and the well of unrefereed OA literature is already available to the same bad actors for poisoning.

But you have a good point in re: biomed research, where the potential bad actors are well-funded, the incentives are large and the potential consequences severe.

A couple of cases of un-reviewed data influencing medical practice in such a way as to cause patient deaths would undoubtedly force new regulations (close down unrefereed OA? legislate disclosure of peer review status?), but it should not come to that.

I have no better answer as yet, but I will work on it.

[MarkD]

I don’t buy the case about copycat drugs. I have been in medicine long enough to know that while one glitizone (for example) may have an adverse affect on a patient another will be tolerated just fine (even though the two are essentially the same chemical entity). There is indeed a societal need for multiple formulations of a single chemical entity. Having said that, a little more innovation in the pharma industry would be in everyone’s interest.

I didn’t comment on OA peer review. I was commenting on something much more fundamental to OA. I was calling into question (in the case of clinical trials) the very concept of making clinical trials publicly available without copyright protection. These papers impact the stock market itself, causing corporate stock price to rise and fall. In such a situation, where billions of dollars are at stake there is a strong financial incentive for people to present the data published in these trials in a favourable light (or unfavourable as the case may be).

In the current regime this incentive is thwarted by copyright law. You cannot disseminate Nature articles (for example) for whatever purpose you wish without their approval. In fact, you must secure their approval and get their approval on how the information will be used and in what context. OA advocates see this as bad, I see this process as a vital check to insure that the material is not inappropriately used for commercial ends. As an OA advocate, how do you propose insuring that OA material is not used inappropriately to advance a private financial agenda? The mechanism in the current regime is copyright. What is the mechanism in OA? That is the point I was trying to make. It is my position that it is socially irresponsible to say that all of this material should be freely accessible without addressing the commercial abuses that could come from that freedom.

[Bill Hooker]

Mark (I hope you're still reading), I've been thinking about this and I'm not clear on the mechanism of the malfeasance that you are worried about. OA does not mean there is no copyright, just that the owner chooses not to restrict access -- so what can be done with OA material that cannot be done with, say, a Nature paper describing the outcome of a clinical trial? One can "disseminate" Nature articles, after a fashion -- you can't *copy* too much, but you can cite them and quote conclusions and so forth. Would it not actually be easier to present the Nature paper in whatever light you prefer, knowing that the original is behind Nature's tollgate? Whenever you quote an OA paper, your audience can go read the original for themselves.