Wikipedia and Digg are both sites which supposedly aggregate the collective wisdom of the internet. However, in reading about recent controversies involving these sites, it became clear to me that most of the content on both sites is produced by a small cadre of elite users and not by the vast majority of people who use these sites. I’d like to talk a little about what the existence of such a “user elite” means for Web 2.0 and what light it sheds on these news stories.
The first story is already a little old, and revolves around the “news” that Wikipedia was establishing new policies over editorial control for certain pages. (It wasn’t true, and the Times eventually added a correction to the online version of the article). The second controversy erupted when Netscape started paying top Digg users to surf the web for Netscape’s new Web 2.0 homepage.
First of all, lets make it clear that neither Digg nor Wikipedia content is produced by “the masses.” It is true that the barriers to entry are far lower than they would be for traditional edited media. In a print Encyclopedia only credentialed and vetted “experts” are allowed to contribute. In a traditional web directory (think the old Yahoo!) users can submit data which is then reviewed by staff before being submitted. In Digg and Wikipedia anyone can log-in and add data. But, not everyone does. In fact, a small cadre of “elite” users produces most of the content on both sites. According to one blogger’s analysis, 56% of Digg’s content is produced by just 100 uers. These users, if only by dint of the amount of time they dedicate to these sites, also have exceptional power over which contributions by other users see the light of day. The vast majority of users are simply “readers.”
This is from the Times story, and quotes Jimmy Wales, the founder of Wikipedia:
The bulk of the writing and editing on Wikipedia is done by a geographically diffuse group of 1,000 or so regulars, many of whom are administrators on the site.
“A lot of people think of Wikipedia as being 10 million people, each adding one sentence,” Mr. Wales said. “But really the vast majority of work is done by this small core community.”
The administrators are all volunteers, most of them in their 20’s. They are in constant communication — in real-time online chats, on “talk” pages connected to each entry and via Internet mailing lists. The volunteers share the job of watching for vandalism, or what Mr. Wales called “drive-by nonsense.” Customized software — written by volunteers — also monitors changes to articles.
This is similar to how blogs work. Most people read blogs. Some people read and leave a few comments. But only a relatively small group of people actually write their own blog. Now, because blogging is free and relatively easy to do, there are millions more people blogging than there are people writing op-ed pieces in their local newspapers; however, this group is still a small portion of the internet community as a whole, and fits within a narrow demographic.
This brings us to the second issue, which is that, because of the narrow demographics of the “user elite,” there exists a systematic bias in the content of these sites, which tend to favor stories about technology and the United States, as well as silly videos.
The second issue came up in Mark Glaser’s coverage of the Digg controversy. He quotes “virtual-reality pioneer, composer, author and tech guru Jaron Lanier“:
[Lanier] worried that sites such as Digg and Reddit were signs of a deeper problem surrounding newsgathering — that we have more news analysts than people on the ground doing hard-nosed reporting.
“It’s true we have a surplus of interpreters of news, as from bloggers, so in a sense we have a gigantic staff of volunteer public analysts, but we are starved for raw data,” he said. “We can read what a blogger on the ground in Israel or Lebanon is experiencing this week, and that is important, but there are almost no unbiased investigative reporters of consequence helping us understand what is going on from a perspective other than that of an ‘ordinary’ person on the ground. This lack is in part a failure of the Internet to serve humanity.”
Lanier then goes a step further, blaming these aggregators for shooting out traffic to silly stories and news of the weird, and ultimately hurting the funding of important, investigative reports.
As insightful as the above quote is, Lanier’s essay “Digital Maoism” seem to reflect a fundamental misunderstanding about how Wikipedia works . He starts by bitching about how inaccurate the Wikipeida page about him is, but never tells us why he doesn’t bother to change it himself? This is a very revealing because it illustrates the confusion I am trying to address here. Much of the hype put out by Web 2.0 enthusiasts suggests that these sites are like a self-regulating insect colony. People like Lanier buy into this hype even as they critique it.
It is important to note that Jimmy Wales was himself never a proponent of such hype. Here is his response to Lanier (whose initial comments are in quotes):
“A core belief of the wiki world is that whatever problems exist in the wiki will be incrementally corrected as the process unfolds.”
My response is quite simple: this alleged “core belief” is not one which is held by me, nor as far as I know, by any important or prominent Wikipedians. Nor do we have any particular faith in collectives or collectivism as a mode of writing. Authoring at Wikipedia, as everywhere, is done by individuals exercising the judgment of their own minds.
Now we turn to the second controversy: Jason Calacanis’s offer to pay select Digg users $1,000 a month to surf the web for Netscape. The genius of this is that Calacanis understands that there is a user elite out there, and he wants them on his site. Unfortunately, I don’t think it will work. It won’t work for the simple reason that the user elite do not exist in isolation from the rest of the Web 2.0 ecosystem. The non-elite users of Digg need to feel comfortable trusting those who do provide content. I personally wouldn’t trust Netscape’s paid-for user elite, and I doubt too many other people will either — even if they were posting to Digg before. There are other reasons as well, such as Digg’s continual innovation in features and design, and the fact that only so many such sites can thrive if they all depend on having a large base of core users.
Another problem with Calacanis’s proposal is that we really don’t need more sites pushing the same crappy news stories at us. As Lanier said, we need better content. Why not pay people $1000 a month to write good original content? You could run a small news bureau on that kind of money!
There have been some serious efforts to gather high quality original news content from the web. Global Voices aggregates bloggers blogging in English about the rest of the world. Newsvine is like Digg for news, but it also allows user-written contributions. And Wikinews is from the creators of Wikipedia. I subscribe to all three and read them regularly, but I still find myself getting most of my news from the Washington Post or from blogs like Cursor.org which read through hundreds of print and online sources so that I don’t have to. The simple truth is that it is one thing to sit at home and read and click, and quite another to walk a beat, make phone calls, and double-check stories — on a deadline. This is hard work that requires a dedicated work force and good editors.
Sites like Wikinews and Global Voices have an increasingly important role to play, but I’m serious when I say that Netscape should be paying people to produce content, not to consume it. We really don’t need more consumers pre-digesting our meals for us…
