The Malleable World (Wikipedia and Knowledge, Part 2)

Diderot began the first encyclopedia riding the wave of excitement that accompanied the enlightenment.  Unlike earlier ages, where determination of the truth relied on revelation or received wisdom, the encyclopedists believed that there was an objective discoverable truth that could be approached through reason.   To say that something was true was to say that it accurately reflected that reality.  The goal of the encyclopedists was to sift through opinion and error and determine was that actual truth was.

Does Wikipedia rest on that same epistemological view of what truth is?

No one that I am aware of, has ever seriously claimed that the traditional way of building reference works is infallible.  (And it annoys me that wikipideans are so eager to punch at that straw man -- are library schools really teaching their students that they should never question traditional reference books?)  What has been argued is that a combination of expert investigation, various modes of peer review, vigorous debate among knowledgeable people, and transparency about inevitable biases is the best way we've found so far to do a pretty good job, and that it has the fallback mechanism of giving the intelligent reader the information that they need to make a judgment about how reliable the information they are getting is likely to be.

Wikipedia suggests that none of these things are necessary.  Not only does it not matter what the expertise of the person drafting an entry is, it doesn't matter whether or not we as the reader know what that expertise is.  We should trust that information (at least) as much as a traditional reference work because the model is self-correcting and errors will (inevitably?) be caught.  And at a certain point, the community will decide that the entry is stable and true.    The underlying bias seems to be that everybody's opinion is as good as everybody else's and the majority will eventually decide what the truth is.

How are we to decide whether or not to believe this?  The implication is that truth is malleable, and that there isn't an objective standard that we should measure information against.   

Try this as a thought experiment -- the most popular tool among residents and young clinicians for assisting their decision making is UpToDate, which rests on the same assumptions of expert opinion and authority as traditional reference works, although it uses digital technologies in creative ways to greatly increase the currency of the information.  And yet, it is undoubtedly rife with errors (it is, after all, a human construct).  Would you be more comfortable having your clinician use it if you knew that it was created using the wikipedia model?

In other words, is the wikipedia way a better approach to discovering the truth about the world, and if so, why?   And what does that say about the nature of truth?

Comments

[Tom Richardson]

I, for one, wouldn't want to rely on a Wikipedia medical knowledge system that linked from, say, Congestive Heart Failure to:

Article not found
From Wikipedia, the free encyclopedia
Jump to: navigation, search

We don't have an article with this title, but you can create it if you log in or create an account. As an unregistered user, you may also submit the content that you wish to have created. Please read our introduction for more information about Wikipedia.

[Marcus]

We must remember that Wikipedia has scaled back its zeal about how it is the best information-delivery system ever devised in the history of mankind. Wikipedia is also seeking experts for scientific topics. They have passed through a messianic phase and are entering a mature phase.

Yes, for medical information there is no doubt that you need trained experts. End of story.

For many other types of content, such as
politics or history, the experts have strong biases themselves, which come through in traditional encyclopedias.

I hear Scott now: "Nobody says the old encyclopedias are perfect. I never learned that in library school. That's a straw man!"

All true. But the Britannica, World Book, et. al, all have a societal "seal of approval" despite their pronounced biases.

Because we must have some authoritative sources to turn to, as a practical matter we discount their biases even if we know intellectually that they exist. This is how I feel that many people approach traditional encyclopedias (but not the readers of this blog). Assuming that this is true--a big assumption--then the Britannica and Wikipedia are not so far apart.

Maybe Wikipedia needs to require signed articles, and/or that a named person be accountable for major changes. The "hive mentality" can still go to work, but now there would be a queen of the hive.

[Mark D]

Marcus, you have a valid point. None of the traditional titles are perfect. I also agree with you that the average reader of the traditional encyclopedia may not be as skeptical as the readers of this blog. But this does not change the fact that the Wikipedia model is subject to gross manipulation on a scale that could never be imagined for traditional titles such as Britannica. Lynn was right I overstated the risk of EBSCO manipulating the Wikipedia entry, but I only did that for rhetorical purposes. The fact is, not all companies are as ethical as EBSCO. Yet the information about EBSCO on Wikipedia was factually inaccurate for many months (so Lynn tells me). However, not all companies (or even interested parties) are as ethical as my good friends at EBSCO. There are those out there who will seek to manipulate Wikipedia content to achieve their own ends. Yes, I know the Wikipedia response, the data cannot be manipulated for long because it is policed by the readers. Sorry, that position does not wash with me. I believe that faith in this self-correcting mechanism is unfounded. Wikipedia can be and most likely is manipulated in a way that would never have been possible with Britannica or any of the other old encyclopedias.

[Faye]

I don't use Wikipedia very often but in online searches it invariably yields results. I recently had an occasion to note some wrong information in an entry. I felt such hubris in correcting the date but it was obviously wrong (according to a standard biography and other, published resources).

I agree with Marcus, as Wikipedia evolves, requiring the author to sign the entry or sign the correction would provide a bit of reliability to the information. Taking the long view of what is preserved and how we created it, the archivist in me wants to know, at the least, when an entry was changed and by whom.

[Mark D]

Yes, signing and dating will help. In my view, that requirement is not enough. Maybe it comes from woking in medical publishing (where nearly every author has some financial conflict), I think more is required. Wikipedia should require signatures, dates, and financial conflict of interests statements. If my friend Lynn makes changes to the EBSCO entry in Wikipedia, I want to know that she is an EBSCO employee.

In the end though, I think there should be a verification mechanism in place at Wikipedia.

[T Scott]

Mark D's last comment sort of gets at the question that I'm wandering around -- why do some people feel the need for a "verification mechanism"? The assumption underlying wikipedia is that the "wisdom of the crowd" is sufficient to supply verification. But why do the proponents believe that this is true? Why are the sceptics uncomfortable with the notion? That, it seems to me, is the underlying question. Are the traditional verification mechanisms a better way of getting to the truth, or just the out-moded structures that we're comfortable with? Arguing about number of errors is beside the point -- I'm trying to get at the underlying epistemological theory.

[MarkD]

Scott, I can’t speak for the origin of the alternative perspective (I don’t wish to call it the opposing perspective because I don’t see this as a them vs. us issue), but I can speak of the origins of my position. Perhaps the best way to approach this is to provide an example illustrating my concerns. Fortunately, this example is actually unfolding as I write this message.

I mentioned in an earlier entry that there is a new phenomena in commercial communication; commercial messages inserted in editorial. The most familiar variation of this is the appearance of a can of coke in a movie storyline. That can of coke isn’t accidental. Most likely Coke paid the producers to include coke in the movie. It is a subtle new approach due to the fact that after decades of bombardment, humans have become adept at filtering out ad messages.

But I digress. I now work for a ‘controlled’ medical publication, meaning that the publication is free to the reader, revenues come from advertisers. I can’t tell you how many times, when I worked for NEJM, librarians who advocated open access would suggest that advertising might be an alternative source of revenue to subscriptions. Problem is, the one who pays the bill, by default, gets the power. So here is my example of why this issue is so critical.

We have just introduced a new service for readers. We will publish a review on a topic. We have published reviews on diabetes, stroke, and dementia. Editorially the object is to give the reader an overall review of the topic in question. How do you diagnose, how do you treat, how do you maintain well-being in chronic illness? Doctors like it because they don’t have to go searching over many issues to get a comprehensive review on a topic. Our business model is open access, doctors get it for free. However these reviews are expensive to create. We seek out the leading authorities to write a review for us. Some of these reviews can run as long as 17 articles. We make our money by getting pharmaceutical companies to sponsor the project. We make it clear who the sponsor is and we allow them exclusive advertising within the review. We also identify any financial conflicts the authors themselves might have. Fair enough then you say, ‘doctors get this information for free, the sponsor is identified, we have a peer review process to insure that the information is accurate and balanced.’ So what’s the problem?

Back to my earlier comment about inserting messages in editorial. I won’t identify the sponsor or the topic – it might give it away as to which pharmaceutical company I am talking about. We are about to publish one of these reviews with our March issue. The product from which we have received support is currently being promoted for a particular condition – in my view (and the view of our experts) it is a bit of a stretch for the product to be promoted for this condition. It offers a minor improvement over competing products and is vastly more expensive. We asked one of the leading authorities in Australia to write the review piece for this particular condition. In the piece he cites data from a BMJ meta-analysis, done several years ago and the last OA paper to be published on the topic. The sponsor has come back to us and and told us that they will withhold funding if we do not change the data from the BMJ meta-analysis to data from their own study which was done five years earlier and published in a less prestigious journal. The BMJ meta-analysis does reference this research, it states in the BMJ article that the results of the study were never subsequently repeated in any other study. It did also mention that the study was well done and contained the largest sample size of any study done on this issue.

In short, the written word means money. If we publish this review with the sponsor’s data the ‘record’ will show that this product has a larger benefit for this condition then we believe (and our author believes) is warranted. Prescription habits will change, more people will get this product then is warranted. This company will see an increase in its sales, it will achieve higher sales then is warranted by patient outcomes. This is better than an advertisement because it is not readily apparent that this is perhaps a biased view, a biased view that favors the sponsor I might add. I have to tell this sponsor today that we will not make the change. I’ve just lost a sale, times are bad here in Australia. We really need that money, it will be painful for us financially. Not everyone would make this choice. And that is the point. There is no control on Wikipedia. There is no one to make these moral decisions. There are many, thousands, tens of thousands, of individuals and corporations out there that have a financial interest in making sure that the information in Wikipedia is positive. As Marcus pointed out, the average reader isn’t going to filter information. My point, the Wikipedia model is ripe for corruption. I have no faith that the mob can correct these biases. How would I know whether or not that piece on EBSCO is correct or not? How would I know whether or not EBSCO truly has the ‘best benefits plan in Birmingham.’ Even if this erroneous information is ultimately corrected, what about the hundreds or even thousands of people who read the incorrect version? The damage is already done.

[Mark D]

From Today's Boston Globe:

"When the news broke last month that US Representative Martin Meehan's staff director admitted deleting unflattering material from Meehan's profile on Wikipedia, the online encyclopedia, it might have been a shock to some. Maybe it shouldn't have been. Wikipedia administrators have since turned up thousands of flattering or disparaging changes in profiles of dozens of members of Congress.
Last week, volunteer investigators discovered that staff members in the office of Senator Norm Coleman, Republican of Minnesota, removed descriptions of him as a ''liberal Democrat" in college. A reference to Senator Dianne Feinstein's payment of a 1992 fine for not disclosing her husband's involvement in her campaign finances was removed by someone in her office.
The revelations that political bias has crept into articles raises new questions about an Internet phenomenon that some are acclaiming as the future of information. And the issues plaguing the site run deeper than political spin. Wikipedia touts itself as ''the free encyclopedia that anyone can edit," and it is exactly that quality that is causing problems."

[Marcus]

Just to add some information--Wikipedia seems to be more "humble" than I believed at least.

Here's a brief passage from the "About Wikipedia" page (http://en.wikipedia.org/wiki/Wikipedia:About):

"Because Wikipedia is an ongoing work to which in principle anybody can contribute, it differs from a paper-based reference source in some very important ways. In particular, older articles tend to be more comprehensive and balanced, while newer articles may still contain significant misinformation, unencyclopedic content, or vandalism. Users need to be aware of this in order to obtain valid information and avoid misinformation which has been recently added and not yet removed."

[MarkD]

Scott, thank you to waking me up about this issue. I had become very complacent about Wikipedia and hadn’t really thought about the damage it was doing. I am ashamed with myself for being so cavalier. It hadn’t occurred to me that that this project was more than just bad publishing and editing. This project can and has destroyed reputations and distorted the truth. This is more than bad editing. It is morally wrong to facilitate character assassination and disseminate falsehoods, and that is in fact what Wikipedia does.

Marcus, I disagree with you, their statement isn’t a sign of humility, rather it is a sign of arrogance and spin. On the face of it, Wikipedia sounds so enticing, freedom of access to information; who could be against that? Freeing information is a noble endeavor. But then there is reality. Though free knowledge is noble, not all thoughts are noble. And that is the flaw in Wikipedia, the fatal flaw. Wikipedia gives equal weight to all thought, to lies and truth, to information and misinformation. To claim, as Wikipedia does, that one should look at its content as a work in progress is not an answer. It is an abrogation of responsibility. When lies are placed on entries or truths removed, Wikipedia treats both as equal. Both are merely classified as historical edits and placed in the history. In so doing, truth deleters may be exposed but myth-makers are then given equal treatment – thus perpetuating and adding credence to the myth. Reputations are destroyed or enhanced at will on Wikipedia.

Those of us who have made publishing our career understand all too well our responsibility. We are the record generators. We understand the power of the written word. Publishers have a responsibility to get their facts straight. Yes, there is bias in traditional publishing. There always has been, there always will be. But to equate bias with out and out propagation of disinformation is an intellectual leap of enormous magnitude. To equate Wikipedia to Britannica (for all its faults and inaccurate information) is like comparing the National Enquirer to The Economist. Britannica never abrogated its responsibility to get its facts straight. Wikipedia has never acknowledged that responsibility. They have cavalierly divested that responsibility to the reader.

[Marcus]

Wikipedia has not abdicated its responsibility. People who post malicious things using it have not behaved responsibly; Mark D should direct his fire at them.

Requiring someone to be accountable for all articles would reduce spurious postings. But ultimately, Wikipedia is a tool, and not a person. Blaming Wikipedia for how people use it is akin to "shooting the messenger." Now we're shooting the software.

There are many problematic aspects of Wikipedia, as this discussion has shown. But in some form or fashion, it seems clear that wiki-like resources are in our future. We have two choices with wikis: 1. Figure out how to make them as good as possible; or 2. Rail against their very existence, and cling to an older reference model that is on the wane.

[T Scott]

Again, I'll go back to the question that I posted at the end of the entry that started this comment thread -- why should I trust Wikipedia? If it is as susceptible to manipulation as Mark D suggests, saying that "wikipedia" is blameless and it is the malicious posters who should be held to account is kind of beside the point. Maybe figuring out how to make wikis better is going to involve moving closer to the model of those traditional reference sources.

[Marcus]

Wikipedia is moving toward the traditional reference model, with the idea of the "stable" article that is harder to update. And they are seeking experts for some articles. This all sounds good to me. Plus, I propose accountable, identified people for all articles (or some subset of "critical" articles), even if modifications by anyone are still possible.

With that said, I think it is important to distinguish between Wikipedia's culpability and that of the people who use it. Blaming Wikipedia for how people use it seems unfair and counterproductive. But acknowledging that Wikipedia does have questionable uses--and some legitimate ones--will hoepfully point us towards necessary improvements.

[MarkD]

Marcus, Wikipedia isn’t a messenger, it is a publisher, and with publishing comes responsibility. That is my point. I am shocked at the irresponsible attitude of the people who run Wikipedia. “Don’t blame us for what people write, we aren’t the authors,’ is not a valid position. Wikipedia employees aren’t the authors but that does not abrogate their responsibility to insure accuracy and to avoid libel and slander. What Wikipedia did to John Seigenthaler Sr. was morally wrong. They allowed people to slander and libel him, they even allowed writers to accuse him of complicity in murdering a US president. What’s worse, they have made it part of the public record. For better or worse, Wikipedia is now part of the public record. Yes, after many months, the good people at Wikipedia, removed the slander and libel from his bio. But so what, they just moved it to the historical archive. It is still a matter of public record. It is still erroneous information, it is still available for anyone to read, and it is still slander.

It isn’t a matter of technology. In the print world anyone could slander and libel in print form, and print has a long shelf life too. The difference is in the Wikipedia founder’s editorial philosophy. They seem think it is OK to have only two people monitor a vast database, they seem to think that it is OK to leave slander and libel and outright lies on their database for months on end. They seem to give equal weight to all writings and to be sanguine about leaving slanderous material on the public record. I’m not a Luddite, this technology is a great thing, it is better than the old print format. However, the Wikipedia editorial model doesn’t hold a candle to the traditional editorial model. Wikipedia isn’t an encyclopedia in the true sense, it is merely a blog pretending to be an enclyclopedia. Wikipedia is most assuredly culpable and that is a fair and productive indictment. I have no doubt that they would most certainly loose in a court of law if John Seigenthaler Sr. decided to press his case.

[T Scott]

I think you're selling the Wikipedians short here, Mark. I don't think they're at all sanguine about errors -- their claim is that, overall, their record of errors will be better than that of a traditional publisher because of the "self-correcting" nature of wikis. The recent Nature article provides some evidence that their error rate may not be that different from Brittanica. The Wikipedia claim is not at all that errors don't matter, but that the vigilance of the community is a better way to root out errors than a reliance on traditional expertise and editing. That's a fairly profound claim about knowledge, and none of the anecdotal examples of errors in Wikipedia disprove it, anymore than Wikipedia's list of errors in Brittanica undermine the claim that a reliance of traditional expertise and editing will produce a quality (if not error-free) product.

[The Other Mark]

I am struck by the similarity in this discussion to the Creationism versus Evolution debate. Traditional encyclopedia publishing is very Creationism--controlled, authoritative, and good results happen relatively quickly. Wikipedia is very Evolution--seemingly random, many mistakes are made, and good results take a relatively long time.

I think that in the future more information resources will come from the Evolutionary model, although a sizeable population will still require the stability of Creationism (or Intelligent Design if you insist). The advantages of the Evolutionary model revolve around the multiple perspectives involved, that hopefully work to create an ecological "best fit." The disadvantages are the many errors that need to be identified and corrected as a topic evolves. The Nature article indicates the Evolutionary model CAN work, but it's a new belief system, and resistance is natural.

Please note this comment is not intended to portray fans of traditional encyclopedia production as mindless fundamentalists. It is merely an analogy that amused me.

[WhippyGoldberg]

The Other Mark: "I think that in the future more information resources will come from the Evolutionary model,..."

Translation: We will devolve into ants with an inability to think for ourselves.