Wikipedia and Knowledge

Marcus and I had an email exchange about Wikipedia while I was at the NLM Long Range Planning Panel meeting last week, sparked by his coming across the Nature article comparing Wikipedia and Brittanica.   He noted some of the changes Wikipedia will be making in order to address some of the legitimate criticisms that have been made, specifically that they'll be introducing a process for tagging an article as "stable" once it has reached a certain "quality threshold" -- that threshold to be determined by having users rate article quality. 

There's an interesting epistemological issue here, and it is much broader than just Wikipedia -- what does it mean for us to know something?   What, in fact, is "the truth"?    The self-correcting principle that Wikipedia relies on could mean: give enough monkeys a typewriter and eventually they will inevitably come up with the truth; or, the truth is simply whatever enough monkeys eventually agree on.  These are very different things.

The principle of "authority" in traditional reference works, now so derided by wikipedia fans, is based on the principle that there is an actual objective truth to be discovered, and that it is best explicated by someone who has made a special study of the case and is willing to put their name to it in order to be tested and to have other experts challenge their findings.   (There is no implication that the "expert" might not get it wrong, and, of course, much of the meat of academic discourse is made up of the disagreements of experts.)  But underlying the "hive-mind" approach to wikipedia is the notion that the truth is whatever enough people agree on.  Is this the wisdom of the hive-mind or the tyranny of the mob-mind?

Consider Wikipedia's discussion of "open access."  While it gets many of the facts right, the tone of the entire article assumes that open access is an unmitigated positive.  There is much on the advantages of open access, with very little discussion of opposing views (the Highwire Press and DC Principles approaches & criticisms are not, for example, even mentioned).  While I personally agree with much of the article, it is ultimately a work of advocacy, not an objective, balanced presentation of the issues.   

So what does that say about "truth"?  If the majority of the wikipedia editors (who've bothered to look at the article) agree with the tone, does that make it true?  I suspect that under a user-rating scheme, the majority of wikipedia readers would be inclined to think the open access article is good, because it supports their own biases (supporters of traditional publishing practices are unlikely to be avid wikipedia users).  So would it be tagged as "stable," i.e., the definitive word? 

In the long run, I completely agree that wiki technology offers the promise of preparing better reference works, more quickly updated, with more of an opportunity for more voices to be heard.  But we need to give more thought to what it means to say that something is true and reliable.  The history of lynching in America, to give just one particularly horrifying example, should give us pause whenever we think to rely on the wisdom of crowds.

Comments

[walt]

Thanks for giving me an even better example of the difficulty in equating popularity with authority (for a mini-essay in an upcoming Cites & Insights)--or "the wisdom of crowds," pretty much the same thing. My other example is that such an equation means that the U.S. (but not the rest of the world) was created a few thousand years ago with all its current species in place, since that seems to be what most people believe. (In the case of this essay, the context isn't Wikipedia--it's Technorati's "authority" ranking, which is purely a popularity measure.)

[MarkD]

I'm a traditional publisher. I use Wikipedia all the time (almost daily). It is useful, it is often (much to my surprize) well written. But I don’t consider it authoritative. You could hardly call, its rule by the mob, a peer review process. Though I think they like to claim that their system equals peer review.

As a heavy user of Wikipedia I would have to classify its content as mixed. The article on John Roberts was excellent. Well researched even balanced. On the other hand, I had a occasion to go to Auckland, New Zealand recently. I went to Wikipedia to read the article on Auckland. Who wrote that piece, the chamber of commerce, the tourist bureau? That is the problem with Wikipedia. You don’t know who writes these articles. You don’t know the author's agenda. To put it in educational terms I give Wikipedia a B-, nice effort even useful. But it needs work. Rule by the mob is not the same as a professional peer review and editing process. For Wikipedia to claim that its method is equal to or superior even to traditional publishing is arrogant and a denigration of the hard work of publishing employees and scholarship.

Then finally, regarding the open access model they use, what happens when their money runs out? It seems to me that this model is ripe for corruption. What if Wikipedia runs into financial difficulty? The new trick in advertising is to hide commercial messages in editorial text. It is a temptation that ‘controlled publications’ (magazines and journals that generate their money solely from advertising) face every day. It would be all too easy to designate as ‘stable’ a piece after an interested party had an opportunity to insert their message. ‘It couldn’t happen,’ you say. I suggest readers go to the entry for the city in which you live. Read it. Many of the community entries you read on Wikipedia are little better than civic booster pieces. Avoid the articles on big cities. Check to see if something is written about your town. I would almost guarantee you that the article will be full of positive stories. See if you can find anything negative. These make great little human interest stories. But I wouldn’t classify it as scholarship.

[janna]

At MarkD's suggestion, I read the entry on San Antonio. Yep, all positive. What's interesting is that, from this article, you would think that the only 'culture' in SA is in bars. It also divides the city into zones/districts/communities, most of which I've never heard called by those names (and I've lived here for almost 22 years!).

On the other hand, I'm delighted to see that Wikipedia has an entry for horseshoe sandwiches, which are much beloved by those of us from central Illinois, but totally unknown to the rest of the world! ;-)

[Marcus]

I've concluded that all "information revolutions" go from a messianic to a measured phase. There should be a graph depicting this evolution; perhaps there is a standard curve for all hot new ideas. Wikipedia has entered a more mature phase, in my view. Obviously further improvements are possible

To pick up on Mark D--No doubt it is true that information about many localities are puff pieces. They reflect the biases of their(unnamed) authors.

But with the Britannica, the biases are just as strong. They just happen to have the sanction of custom.

As an example, I would like to know how the Britannica treated Mahatma Gandhi in the 1930's (when he was a threat to the British Empire) and how they treat him today. For that matter, what did the next Brittannica say after the Oscar-winning movie was released? How has that changed by now?

For my money, the standard reference sources reflect the prejudices of their times. Perhaps this is better than the wisdom of crowds, but it is a weakness all the same.

Librarians may know that different sources have different strengths. But all the standard texts have an imprimatur of "correctness," which is often not justified.

[Tom Roper]

I remember a library school exercise I did on the postgraduate course at the then Polytechnic of North London which I attended in 1983.
We had to choose a subject and see how it was treated in different encyclopaedias. I took Fidel Castro as my subject and compared his treatment in the Britannica, the Americana and the Great Soviet Encylopaedia (Bolshaya Sovetskaya Entsiklopediya). As you might have predicted, the entries varied wildly in their assessment of Fidel. But they did not even agree on simple basic facts: all three gave different dates of birth for example. So I agree with Marcus.

[Lynn]

I followed Janna's suggestion and looked at the entry for Birmingham, Alabama. I was surprised to see that someone has informed the wiki-world that EBSCO Publishing is based in Birmingham. I'm sure the hundreds of people who work for EBSCO Publishing are getting tired of that long commute from Ipswich MA...which is where EP is actually headquartered. EBSCO Industries (the parent company) is located in Birmingham. The entry for EBSCO Industries was corrected recently after I asked our Corporate Communications department to look into it. I found several obvious mistakes - including the name of the company's founder. These mistakes would not have been made by anyone knowledgeable about the company. The advantage to Wikipedia is that, unlike a print resource, those kinds of mistakes can be corrected. The disadvantage is that nobody knows if mistakes have been corrected or not.

Yes, Wikipedia contains a wealth of information and is often a good place to start. But woe-be-it to the typical high-schooler who uses it to research his term-paper. The same is true for those "old fashioned" encyclopedias, which is why most of my high school teachers allowed their use ONLY in conjunction with other resources. God bless 'em...

[Mark D]

Marcus, we agree completely. My point isn't that the traditional model was unbiased and Wikipedia is biased. My point is; I know the Britannica bias before I even get started, same goes for the Yanks and the Soviets. The problem with Wikipedia is that all are involved and no one is involved. I don't know how to discount bias at Wikipedia. I'm not complaining that Wikipedia is biased, I'm complaining that I have lost my reference point in discounting that bias. The ironic exception, the Wikipedia piece on open access. It's bias on that topic is as I would expect it to be.

Lynn, also proves my point in her piece. Do we really want EBSCO employees editing what is said about EBSCO in the Birmingham entry? If that is the case, Mr Stevens will go down in Wikipedia history as a saint. Not that he wasn't a saint, its just if the EBSCO employees edit the entry it isn't likely to be a balanced view of EBSCO. Sorry Lynn, but even EBSCO has to have its sins. There is nothing wrong with this approach as long as I know EBSCO had a hand in it. But, except for the small group of people that read this blog, no one knows that EBSCO employees had any input into the entry as it now appears on Wikipedia.

[Lynn]

Mark nails it - my first concern with Wikipedia is that entries are anonymous. My second concern is that some people who read it evidently can't distinquish between facts and opinion. In the presence of incorrect facts, one cannot possibly present a "balanced view". Well, unless you are a participant in the current administration (pop quiz - is this statement fact or opinion?)

Don't primary schools teach the difference between facts and opinion any more? Do intellegent, well-educated people no longer care that there might be a distinction?

I disagree with Mark on the unlikelihood of someone intimately familiar with a topic being able to present a balanced view. The original entry says: "EBSCO is also well known for its employee benefits." This statement was news to me. By what measurement? It doesn't say that we are well known for having exceptionally good benefits or well known for having louzy/limited benefits. But I let the statement stand. That's a bias.

The original entry spells our founder's name STEVENS (just as Mark did). It is STEPHENS. The original entry declared that he founded in the company in 1943 while he was a law student. In fact, he graduated from law school in 1936 and he and his wife founded the company in 1944. Those are verifiable facts; they are not biases.

There were several other verifiable, factual inaccuracies, that anyone who might bother to type www.ebsco.com and poke around would discover. I think it is expected that almost all websites will have biases, but whomever wrote the original EBSCO entry didn't even bother to verify the FACTS. They evidently just typed in what bits they remembered from god knows what source. And that's another thing - at least some encyclopedias have brief bibliographies indicating the origin of the information. Wouldn't that be useful?

Oh, and as a PS to correct Mark's bias that EBSCO employees editing what is said about EBSCO in the Birmingham entry would result in Mr Stevens going down in Wikipedia history as a saint - not even the most loyal EBSCO employee would depict Elton B. Stephens as a saint. With respect to Mr. Elton and all he accomplished, if "saint" ever shows up in the EBSCO entry, you can be absolutely sure it was edited by someone who is not an EBSCO employee.

[John Beekman]

While I have not looked at the city entries mentioned, I do know that there was huge spike in the number of Wikipedia entries in (I think it was) 2003 when a "bot" used (again, I think) census data to create entries on towns and cities. This may explain some of what people see in those entries.

Discussions of this and other aspects of Wikipedia can be found in the "meta-wiki" pages: http://meta.wikimedia.org/wiki/Main_Page
Great data for the curious or for an MLIS student writing a paper about the phenomenon.

[The Other Mark]

An interesting feature article from Information Today:

Wikipedia and Britannica
The Kid’s All Right (And So’s the Old Man)
by Paula Berinstein | Consultant, Berinstein Research

http://www.infotoday.com/searcher/mar06/berinstein.shtml

[Lynn]

The followers of this blog, especially those who participated in the wikipedia discussion, will be interested to read the Encyclopedia Britannica's 20 page rebuttal to the Nature article at http://corporate.britannica.com/britannica_nature_response.pdf