Wikipedia’s reliability?

This morning, the WSJ ran a story on the Encyclopedia Britannica’s efforts to defend itself against assertions that it’s no more accurate than Wikipedia: WSJ.com – In a War of Words, Famed Encyclopedia Defends Its Turf

Here’s the Nature study on Wikipedia vs Encyclopedia Britannica that kicked the whole thing off back in December.

A month ago I had discussed the topic with my dad, a cognitive psychologist who has spent a lot of time studying the scientist Michael Faraday. He was pretty unimpressed with Wikipedia, and did a comparison of its entry on Michael Faraday with Britannica’s. Here are Dad’s comments:

Yeah, I’d known about the “Nature” study, but it’s really pretty poor, IMHO.

First of all, they used only one expert per pair of articles. Wouldn’t surprise me if most of them could come up with pretty good guesses about which was which (this would probably work in Wikipedia’s favor, but it’s still a bias). Even so, one per pair is too few.

Second, that’s a pretty puny sample size of articles. At a maximum of 39 cents per letter ( x 50 = $19.50), assuming they used snail mail, and some xeroxing, they could have done better for very little effort!

I redid my own short test (one I’d done a while ago) by looking up the articles on Michael Faraday in Wikipedia and in Britannica Online. The levels are very different, of course; the EB one is much longer and has some very abstract stuff in it. Even so, the comparison is startling.

Wikipedia’s is riddled with error, apparently most of it being based on a recent “fictionalized” biography of Faraday that simply invented stuff about his motivations and that of other people, events that “must” have happened, conversations that “probably” occurred, and so on. Of course the qualifiers all disappeared in the article, and I now see, for example, Faraday credited as having invented the Bunsen Burner as his major achievement. What nonsense! Bunsen’s device was indeed based on Faraday’s research on flames, but he no more invented the burner than he invented the “Faraday Flashlights” now being sold all over eBay.

There are other mistakes even more serious, but I won’t bore you. The mistaken stuff about electricity, magnetism, and chemistry is frightening, never mind the hash made of the history. The links to other sources are just as bad — poorly chosen crap, for the most part.

The EB article was written by Pearce Williams, a major guy in the field. It was written some 15 years ago, so it errs on some small scholarly points, since resolved differently, but it has a masterful account of the science, and of Faraday’s life. The only big “error” is the attribution to Faraday of a strong influence by Boscovich. Again, not to bore you, but this was a hobby horse ridden by Williams for years, one that most scholars (me included) now think was overdone or flat wrong. But ’twas a heated controversy for a while, so one can perhaps forgive in retrospect. A major error, if that’s what it is, only to nitpicky historians. If I’d read this “blind”, I’d have guessed Williams was the author, I’m sure.

The EB article has surprisingly full references (but to heavy scholarly stuff, print media only) and links to good serious articles in the EB covering the physics and chemistry. Here again, a different level altogether — you wouldn’t send the average high school student to the EB for this. But if sent to Wikipedia, they’d of course be able to fully understand all the horseshit given there.

On balance, EB, hands down! You get what you pay for in this case!

I asked Dad why he didn’t just do the public-minded thing, and spend some time correcting Wikipedia’s egregious errors? From his response:

The EB article on Faraday was much longer than the one in Wikipedia, a factor of three, maybe. So that means there was 3 x the opportunity for an error in the EB article. If the sample of 50 entries in the Nature study was similar (i.e., longer EB articles), then the number of errors would really be skewed in Wikipedia’s favor. The way around this problem would be to compute the # errors per 1000 words, or some such. But then they would have had to be thinking like a scientist!

… Been mulling your point about correcting the Faraday article in W. Problem is, I’d have to write a whole new article! It’s not just a matter of changing factoids. Having just spent a month writing a bio of a psychologist for the “Dictionary of Scientific Biography” (for which Scribner’s is paying me! Hey, I’m a free lancer!), I know how much work would be involved to do it right.

I’ll keep mulling, but there is also the problem of whether those of us who do have some expertise should enhance an endeavor that is perpetuating misinformation, partial information, whatever. Thus, I looked at the Wikipedia article on Jonathan Edwards. Seems mostly o.k. factually (one minor error), but says nothing at all about his philosophical contributions, nor even the fact that he made any. His “great work on the Will” is mentioned, but no clue what it’s about, other than the book’s title. You do get links to lots of theological figures he was associated with (but not his most important predecessors), but the article is really about Edwards as a pastor. As with Faraday, the EB article is about 3 x longer, covers the topic very well, including the philosophical issues, as well as the theological. It’s a signed article by a real person (I don’t recognize the name, but he knows his stuff).

Wikipedia’s reliability?
  • mathwiz2020

    Hi, I’m a frequent editor of Wikipedia and an admin on the site. I am particular interested in your comments because I myself edited the article on Faraday just about a year ago. I based most my edits off of the book “A life of discovery: Michael Faraday, giant of the scientific revolution” by James Hamilton. It seems like a nonfictional biography if you look at the comprehensive list of sources in the back, but that may not be the book you’re referring to.

    The beauty of Wikipedia over Britannica, in my opinion, is the aspect of user contribution. Let’s say someone makes a mistake. Well, I can notice the mistake and correct it! If someone makes a mistake in EB, the editors aren’t going to notice it and correct it. Wikipedia depends on community interaction. If you see an article you think is junk or entirely incorrect, feel free to change it or even completely rewrite. One of Wikipedia’s policies is “Be bold!”

    Just write a credible article, cite it with a nice bibliography or web sources, and no one will mind. It’s like community service – but better. Contributing to the world’s largest free encyclopedia produces a feeling of self-satisfaction and, most importantly, lets you hone in on your passions, the subjects you’re interested in, and help the world learn from you.

    Please, I encourage you to use your knowledge for good. Edit Wikipedia’s Faraday article, and it will do us all good. Thank you.

  • mathwiz2020

    By the way, if you want to contact me, you can leave a message at my Wikipedia talk page.

  • http://dylan.tweney.com Dylan

    Thanks for your comment, Mathwiz.

    You’ve basically validated my (and my father’s) point. Britannica’s entry on Michael Faraday was written by a serious scholar of the field who cites dozens of sources. Your entry was written after reading one popular book on Faraday (and an error-prone one, to boot).

    Sure, the experts could spend time rewriting and correcting the entry, but what’s in it for them? Unless an expert gets a frisson of good feeling from contributing anonymously to the social good of Wikipedia, nothing. Britannica at least pays — a pittance perhaps, but it’s something — in exchange for expertise.

    The bottom line is that Wikipedia is good in the areas where people who know what they’re talking about have taken the time to do research and write careful entries. Where they haven’t, it’s not. The problem is not how do we make Wikipedia better — the answer to that is obvious — but how do you *trust* Wikipedia, if you can’t tell the difference between the two kinds of entries?

  • http://orderofr.net Michael Bunsen

    Thanks for sharing your discoveries and comments.

    I think “contributing to the social good” is a pretty good reason. Rather than maybe donating to Amnesty International or calling a senator, people with expertise have an amazing opportunity to contribute to the global social good by simply contributing to Wikipedia. Writing a few summarizing paragraphs on Wikipedia in ones specific field is an easy task that should be a second-nature, at-home task to anyone writing or doing research in a subject.

    Secondly, I think it’s right to question every word presented in Wikipedia. Although I use the source extensively as a “jumping point” to get an instant gist of topics and find related research breadcrumbs, I never make it my only source.

    …And maybe there should be a special edit or rating button on Wikipedia along the lines of, “This information is inaccurate to my knowledge but I don’t have time to correct it” :)