I first met Jimbo Wales, the face of Wikipedia, when he came to speak at Stanford. Wales told us about Wikipedia’s history, technology, and culture, but one thing he said stands out. “The idea that a lot of people have of Wikipedia,” he noted, “is that it’s some emergent phenomenon — the wisdom of mobs, swarm intelligence, that sort of thing — thousands and thousands of individual users each adding a little bit of content and out of this emerges a coherent body of work.”† But, he insisted, the truth was rather different: Wikipedia was actually written by “a community … a dedicated group of a few hundred volunteers” where “I know all of them and they all know each other”. Really, “it’s much like any traditional organization.”
The difference, of course, is crucial. Not just for the public, who wants to know how a grand thing like Wikipedia actually gets written, but also for Wales, who wants to know how to run the site. “For me this is really important, because I spend a lot of time listening to those four or five hundred and if … those people were just a bunch of people talking … maybe I can just safely ignore them when setting policy” and instead worry about “the million people writing a sentence each”.
So did the Gang of 500 actually write Wikipedia? Wales decided to run a simple study to find out: he counted who made the most edits to the site. “I expected to find something like an 80-20 rule: 80% of the work being done by 20% of the users, just because that seems to come up a lot. But it’s actually much, much tighter than that: it turns out over 50% of all the edits are done by just .7% of the users … 524 people. … And in fact the most active 2%, which is 1400 people, have done 73.4% of all the edits.” The remaining 25% of edits, he said, were from “people who [are] contributing … a minor change of a fact or a minor spelling fix … or something like that.”
Stanford wasn’t the only place he’s made such a claim; it’s part of the standard talk he gives all over the world. “This is the group of around a thousand people who really matter”, he told us at Stanford. “There is this tight community that is actually doing the bulk of all the editing”, he explained at the Oxford Internet Institute. “It’s a group of around a thousand to two thousand people,” he informed the crowd at GEL 2005. These are just the three talks I watched, but Wales has given hundreds more like them.
At Stanford the students were skeptical. Wales was just counting the number of edits — the number of times a user changed something and clicked save. Wouldn’t things be different if he counted the amount of text each user contributed? Wales said he planned to do that in “the next revision”, but was sure “my results are going to be even stronger”, because he’d no longer be counting vandalism and other changes that later got removed.
Wales presents these claims as comforting. Don’t worry, he tells the world, Wikipedia isn’t as shocking as you think. In fact, it’s just like any other project: a small group of colleagues working together toward a common goal. But if you think about it, Wales’s view of things is actually much more shocking: around a thousand people wrote the world’s largest encyclopedia in four years for free? Could this really be true?
Curious and skeptical, I decided to investigate. I picked an article at random (“Alan Alda”) to see how it was written. Today the Alan Alda page is a pretty standard Wikipedia page: it has a couple photos, several pages of facts and background, and a handful of links. But when it was first created, it was just two sentences: “Alan Alda is a male actor most famous for his role of Hawkeye Pierce in the television series MASH. Or recent work, he plays sensitive male characters in drama movies.” How did it get from there to here?
Edit by edit, I watched the page evolve. The changes I saw largely fell into three groups. A tiny handful — probably around 5 out of nearly 400 — were “vandalism”: confused or malicious people adding things that simply didn’t fit, followed by someone undoing their change. The vast majority, by far, were small changes: people fixing typos, formatting, links, categories, and so on, making the article a little nicer but not adding much in the way of substance. Finally, a much smaller amount were genuine additions: a couple sentences or even paragraphs of new information added to the page.
Wales seems to think that the vast majority of users are just doing the first two (vandalizing or contributing small fixes) while the core group of Wikipedians writes the actual bulk of the article. But that’s not at all what I found. Almost every time I saw a substantive edit, I found the user who had contributed it was not an active user of the site. They generally had made less than 50 edits (typically around 10), usually on related pages. Most never even bothered to create an account.
To investigate more formally, I purchased some time on a computer cluster and downloaded a copy of the Wikipedia archives. I wrote a little program to go through each edit and count how much of it remained in the latest version.† Instead of counting edits, as Wales did, I counted the number of letters a user actually contributed to the present article.
If you just count edits, it appears the biggest contributors to the Alan Alda article (7 of the top 10) are registered users who (all but 2) have made thousands of edits to the site. Indeed, #4 has made over 7,000 edits while #7 has over 25,000. In other words, if you use Wales’s methods, you get Wales’s results: most of the content seems to be written by heavy editors.
But when you count letters, the picture dramatically changes: few of the contributors (2 out of the top 10) are even registered and most (6 out of the top 10) have made less than 25 edits to the entire site. In fact, #9 has made exactly one edit — this one! With the more reasonable metric — indeed, the one Wales himself said he planned to use in the next revision of his study — the result completely reverses.
I don’t have the resources to run this calculation across all of Wikipedia (there are over 60 million edits!), but I ran it on several more randomly-selected articles and the results were much the same. For example, the largest portion of the Anaconda article was written by a user who only made 2 edits to it (and only 100 on the entire site). By contrast, the largest number of edits were made by a user who appears to have contributed no text to the final article (the edits were all deleting things and moving things around).
When you put it all together, the story become clear: an outsider makes one edit to add a chunk of information, then insiders make several edits tweaking and reformatting it. In addition, insiders rack up thousands of edits doing things like changing the name of a category across the entire site — the kind of thing only insiders deeply care about. As a result, insiders account for the vast majority of the edits. But it’s the outsiders who provide nearly all of the content.
And when you think about it, this makes perfect sense. Writing an encyclopedia is hard. To do anywhere near a decent job, you have to know a great deal of information about an incredibly wide variety of subjects. Writing so much text is difficult, but doing all the background research seems impossible.
On the other hand, everyone has a bunch of obscure things that, for one reason or another, they’ve come to know well. So they share them, clicking the edit link and adding a paragraph or two to Wikipedia. At the same time, a small number of people have become particularly involved in Wikipedia itself, learning its policies and special syntax, and spending their time tweaking the contributions of everybody else.
Other encyclopedias work similarly, just on a much smaller scale: a large group of people write articles on topics they know well, while a small staff formats them into a single work. This second group is clearly very important — it’s thanks to them encyclopedias have a consistent look and tone — but it’s a severe exaggeration to say that they wrote the encyclopedia. One imagines the people running Britannica worry more about their contributors than their formatters.
And Wikipedia should too. Even if all the formatters quit the project tomorrow, Wikipedia would still be immensely valuable. For the most part, people read Wikipedia because it has the information they need, not because it has a consistent look. It certainly wouldn’t be as nice without one, but the people who (like me) care about such things would probably step up to take the place of those who had left. The formatters aid the contributors, not the other way around.
Wales is right about one thing, though. This fact does have enormous policy implications. If Wikipedia is written by occasional contributors, then growing it requires making it easier and more rewarding to contribute occasionally. Instead of trying to squeeze more work out of those who spend their life on Wikipedia, we need to broaden the base of those who contribute just a little bit.
Unfortunately, precisely because such people are only occasional contributors, their opinions aren’t heard by the current Wikipedia process. They don’t get involved in policy debates, they don’t go to meetups, and they don’t hang out with Jimbo Wales. And so things that might help them get pushed on the backburner, assuming they’re even proposed.
Out of sight is out of mind, so it’s a short hop to thinking these invisible people aren’t particularly important. Thus Wales’s belief that 500 people wrote half an encyclopedia. Thus his assumption that outsiders contribute mostly vandalism and nonsense. And thus the comments you sometimes hear that making it hard to edit the site might be a good thing.
“I’m not a wiki person who happened to go into encyclopedias,” Wales told the crowd at Oxford. “I’m an encyclopedia person who happened to use a wiki.” So perhaps his belief that Wikipedia was written in the traditional way isn’t surprising. Unfortunately, it is dangerous. If Wikipedia continues down this path of focusing on the encyclopedia at the expense of the wiki, it might end up not being much of either.
Excellent article.
posted by Paul Simpson
on September 4, 2006 #
An excellent article. I tried to vote, but since I am one of your “occasional contributors” (I’ve edited only one article to make content changes), I am not eligible to vote. It appears that the opinions of “occasional contributors” will not be heard.
posted by Eric
on September 4, 2006 #
Great article, Aaron. Will you release your code and conclusions?
posted by Cory Doctorow
on September 4, 2006 #
I need to clean the code up before I can publish it, but I’m happy to work with anyone who’s interested in pursuing this further.
posted by Aaron Swartz
on September 4, 2006 #
Andrew Gelman says “I suspect that the clean and uniform format is a large part of Wikipedia’s air of authority.”
Spot on, BUT the “air of authority” is quite a different matter than actual “authority”.
The good sign is that format, etc. sells tickets but content changes the world.
The bad sign is that those who groom this monkey seem to have taken control of the project by setting themselves up as a hierarchy via voting eligibility rules.
The people Aaron shows to have crafted the content are largely disenfranchised while the “eyeshade editors” rule.
Hope I’m wrong.
Love.
posted by William Loughborough
on September 4, 2006 #
“Don’t worry, he tells the world, Wikipedia isn’t as shocking as you think. In fact, it’s just like any other project: …”
And he’s correct, though that cuts both ways. There’s an “editorial staff” which handles the copyediting, versus the occasional freelance contributors, who work on specific topics. That is indeed a standard model.
What’s really interesting about Wikipedia is how:
1) It manages NOT TO PAY almost anyone, getting people to donate their services, on the basis of “community” or other ideas.
2) It’s developed elaborate intellectual defenses against the failings of #1, whenever the flaws show through.
Those are two aspects which are, if not exactly completely novel, definitely innovative implementations of eternal themes.
Let’s again recall there’s $4 million of venture capital invested in the associated Wikia venture, and at a rule of thumb of 10x return, the investors want at least $40 million. That’s got to come from somewhere, and figuring out how to get work for free seems like a major part of it.
posted by Seth Finkelstein
on September 4, 2006 #
Seth, you seem to think those venture capitalists are being rational in expecting Wikia to succeed to the tune of $40 mill; I’d say that that is rather optimistic and reminds me not a little of the first Internet bubble.
posted by maru
on September 4, 2006 #
You are obviously speaking of good encyclopedias. I had a wild boss who, in his earlier years, got an editorial job with one of those grocery store encyclopedias. He specialized in R through S.
Julie
posted by julie
on September 4, 2006 #
“Observe: The humorless wikipedia administrator in its natural habitat!” - me, looking at the ‘recent changes’ page of the Encyclopedia Dramatica
There are the people who contribute content, and there are the people who maintain it. The former doesn’t know the intimate details of the wiki’s features. The latter does, and will sometimes leave snippy comments about the lack of this as they make it conform to the spec.
The mindset is the same, whatever wikipedia-type place it’s found on. Even in ones dedicated to misinformation and gossip.
posted by Egypt Urnash
on September 4, 2006 #
Very insightful Aaron.
In fact to think of it, I guess it is this flawed understanding of the nature of the contribution process to Wikipedia which Wales harbors might be behind the recent decision by Wikipedia to ban edits by users who are not logged in.
I think this is one of the most regressive moves in the history of Wikipedia - by putting in a login ‘barrier’ i would assume a lot of the casual (but as you pointed out, very critical) contribution is put off.
This type of casual contribution is at the heart of Wikipedia, and no amount of vandalism is reason enough to put barriers around this.
It is like killing your goose which lays the golden eggs, just because you cant stop some of them from being stolen.
posted by neeraj
on September 4, 2006 #
I already voted, and now I’m annoyed I didn’t vote for you!
<
p>This is a fantastic observation. If you weren’t running in the Board election, I’d be spreading it far and wide. As is, I feel too far into the guts of Wikimedia politics to be spreading it with propriety and not appearing to push your candidacy per se … gah.
<
p>But yes. This has major implications for policy and how Wikipedia works.
<
p>(And it’s also a nice big stick for me to hit people over the head with concerning Byzantine overengineered processes.)
posted by David Gerard
on September 4, 2006 #
David,
You can vote again, and it will override your previous vote.
posted by Jacob Rus
on September 4, 2006 #
What computer cluster did you use? What kind of programming environment is it?
posted by Will Bendick
on September 4, 2006 #
Aaron - this page was noted on wikien-l, and I suggested to Gregory Maxwell that you two should talk about how to get good data across the whole database. Greg is very good at running interesting queries on the whole database - if your methodology can be automated, this will give you accurate data. Since if you’re running for the Board because of this, I assume you’d prefer greater certainty!
Thread starts here: http://mail.wikipedia.org/pipermail/wikien-l/2006-September/thread.html#53307
posted by David Gerard
on September 4, 2006 #
i love wikipedia.
but i have felt that my contributions
— on topics where i do have expertise —
have been warped by people who merely want
to “make it sound like an encyclopedia”
without having any knowledge of the topic.
it seems to me that the experts that
wikipedia so desperately needs will be
precisely those people who are not
making edits to a large number of topics.
-bowerbird
posted by bowerbird
on September 4, 2006 #
Interesting article, but I think you’re being overly dismissive of the insiders/formatters. Because the likelihood of casual users making significant contributions is determined not only by how easy it is for casual readers to contribute, but also by how many such readers there are to begin with. And the size of that pool of readers depends to a great extent on how readable, reliable and accessible the information presented is. So a change that imposes a minor impediment to casual edits can still result in a net increase in casual editors if it also helps the “insiders and formatters” improve Wikipedia’s usability, and increase its readership.
posted by Abou Ben Adhem
on September 4, 2006 #
I second what bowerbird says. I used to be one of the people making small but regular contributions to Wikipedia, on the subject of places I have lived and the London theatre scene/history, both subjects on which I have a lot to contribute, but got fed up of the self-appointed officious jobsworths who would come along minutes later and revert your contribution or substantially modify it (so that it lost the important detail), in order that things would fit “their vision” for the sections they “patrol” and have “made their own”. My time is too valuable to argue with these people, sit monitoring discussion pages, or fight for ages to wrestle control back (and in doing so become as bad as these people, if I were to end up spending my entire waking hours on Wikipedia as they appear to).
I’ve also become annoyed with the behaviour of this perceived “inner circle” when it comes to current events and the “high and mighty” attitude they have to outsider contributors (not me in these cases).
Wikipedia was good up until 12-18 months ago but lately it’s become far too cliquey. Which scares a lot of knowledgeable would-be contributors off, because if we are to invest our valuable time contributing some expert knowledge on some subject, we want to know that our work will remain there for others, and not just keep getting reverted out in seconds by some control freak that knows nothing about the actual subject.
Why am I taking the time to make this point here? Because your article proves the existance of this “inner gang” that I feel are actually holding Wikipedia back. To allow Wikipedia to grow and really pick the brains of the experts around the world, you need to do something to break up this inner gang and the mini empires they are building for themselves.
posted by Ian
on September 4, 2006 #
As an “occasional contributer” I’ve found largely the same thing. The “super editors” are the ones whom the site benefits, and, unfortunately, they share the same opinion as Wales that they are responsible for the bulk of the content. I’ve proposed the addition or deletion of a couple articles, cited Wikipedia policy (two or three points in each case) and have been told point blank that I am wrong and that “the other editors would agree with” the editor I was arguing with. I may not spend 10 hours a day on the site or have thousands of edits under my belt, but this sort of treatment just left me with a bad taste in my mouth, and it sounds like the founder is all but condoning it.
posted by Anonymous
on September 4, 2006 #
All of the original content before it was switched to PHPWiki was done by an almost entirely different group than the people who are doing it today, so while there may be 500 active supercontributors at any one time, between 2001 and now there has been a constantly changing group of active contributors. This has an effect on both the numbers that are being used here, I think. Early on, there was only about 500 people, total contributing, but everyone was a supercontributor.
As a side note, it was much easier to contribute before, when you could discuss with other contributors and know that someone really had to be disruptive to be blocked. It happens way too easily now, primarily I think because of a number of truly unprofessional so-called “moderators”. The power imbalance now is very anti-wiki.
alan_dershowitz (one of the earlier contributors.)
posted by alan_dershowitz
on September 4, 2006 #
This is a smart and important post.
posted by Petey
on September 4, 2006 #
Dear Aaron
As another occasional contributor I’ve established 2-3 articles and watched them get a life of their own.
When I tried to vote there was a curious response:
Sorry, you are not qualified to vote in this election here on the English Wikipedia. You need to have made 400 edits here before 2006-08-01T00:00:00; you have made 31. Also, your first edit on this wiki was at 2005-04-29T05:57:45; it needs to be before 2006-05-03T00:00:00.
Curious because in my understanding 2005 is before 2006. Perhaps I am misunderstanding the rule?
Good luck with the election.
Simon
posted by Simon Pockley
on September 5, 2006 #
Regardless of how many people contribute to Wikipedia content, I remain concerned about the unchecked biases of contributors. Anyone can can make impressive-sounding citations to create an air of authority when writing/editing a topic. But what percentage of those occasional, unregistered contributors who are subject matter experts on that topic review new contributions to verify the changes? While this may be trivial for celebrity bios, it is particularly important for controversial scientific entries, such the standard model for physics, xenoestrogens, and global warming.
At least with a traditional encyclopedia publishing process, the compiling company hires people with well-established expertise to check the work of free-lance contributors. And on the most important topics, the compiling company gets top-notch subject matter writers to begin with. None of this is guaranteed in cyberspace.
As a test of my theory the problems with the lack of peer review in Wikipedia, I’ve augmented a topic (with citations) on a controversial medical topic to see if anyone bothers to challenge my assertions. It will be interesting to see the revisions of that article.
posted by jake3_14
on September 5, 2006 #
Great work, great post. I would vote for you if I was in the 400+ club.
posted by david mathers
on September 5, 2006 #
Your point is well taken, with a caveat: that it’s difficult to actually quantify particular anonymous users’ contributions to Wikipedia.
As anonymous users are tracked by IP address, any one real-world user may be broken up into a countless Wikipedia ‘users’ if they happen to reside behind a dynamic IP.
Even given a fairly stable IP address here at home, the Wikipedia user contributions page for my latest anonymous ‘identity’ only lists 23 edits to my name. Another one from a month or two back lists only 6. My IP address at work seems to switch on an almost hourly basis, so multiple casual edits from even a single day from there may show up as isolated edits by random readers passing in the night.
The idea that the bulk of Wikipedia’s substance comes from the more heavily specialized, non-community users strikes me as inherently reasonable (and actually quite reassuring).
I don’t think, though, that you can establish with any certainty whether these anonymous users return time after time or if they really just make a handful of changes and never hit ‘edit’ again.
posted by Dan
on September 5, 2006 #
This is fantastic research and very actionable. Unfortunately, as you note, some actions are going the other way (making it harder for the less frequent but more knowledgeable users).
The other major grip I have with Wikipedia recently is the proliferation of citations. I understand why people think they are of value but I feel like a) they lead to another false sense of authority and b) they lead to disjointed articles (ie., I’ve seen articles that seem like a long list of citations with not a single sentence seeming to flow from the preious one).
posted by pwb
on September 5, 2006 #
I wrote several wikipedia articles, where I wrote an entire article up out of nothing. In each case, the article was then “edited” — without any notification to me — by some super-editors, who removed content, and turned what I thought was gosh-darn good writing into crap. I tried to explain to friends my frustrations, and they told me “Why do you even bother? Of course it’s just a bunch of self-important snobs.”
So I went and looked at the people who “edited” my contributions, and saw that my friends were right. These people, by and large, “edited” thousands of articles. In most cases, these edits were to remove material that they found unsuitable. Indeed, some of the people-history pages contained little “awards” that people gave each other — for removing content from Wikipedia.
Since I heard of your article, I headed on to the voting page, and, alas and alack, I am not eligible to vote, since I have only made 21 article edits. I was also informed that since I didn’t have the required number of edits by yesterday, I cannot become eligible.
HURRAH, I am DISENFRANCHISED.
All this leads me to ask, “I wonder if those little “awards” that people give each other count towards the required number of edits?
There’s an old saying, “The lower the pay, the higher the politics.” I’ve seen it happen at college student activities, non-profit volunteer community organizations, etc.
I believe it’s happening here, now, with Wikipedia.
posted by Bill Coderre
on September 5, 2006 #
Perhaps a more useful metric would be: # of letters typed that are still present in the current release. This would weed out those who contribute garbage data that is only removed later. An even more useful but difficult metric to measure would be: # of letters typed that are present in the most recent non-garbage release (as the current version of some pages may be vandalized).
posted by Liam Morley
on September 5, 2006 #
did anyone check how many of Wales’s top .7% of most contributing authors are actually bots?
posted by tarkowski
on September 5, 2006 #
I think that this is very insightful and reveals the great ‘black hole’ that wikipedia could be drawn into. If I had the chance to vote, it would be for you. I fully support your campaign and would be interested to hear the outcome.
posted by SonicSpeed
on September 5, 2006 #
I’ll join in saying: I’d vote for you, but I’m well under 400 official edits. I’ve made a lot more than my credited ~40, but I don’t usually log in. I wouldn’t have thought it was important…
posted by Jeremy Dunck
on September 5, 2006 #
I find it funny that people are complaining it’s not easy enough to make edits because the big bad super editors revert your writing. Look — wikipedia can’t have it both ways, you need to cite your facts, sometimes that means going to the library to cite a fact you already know to be true. Maybe a google book search, a news search, whatever.
Has anyone noticed the articles about wikipedia in the news lately? They are all scare tactics — wikipedia is a fraud because anyone can make edits! That’s why the editors have to be so tough about what is added, if they really did allow any old thing to be added wikipedia would be completely useless.
So stop complaining, we can’t have it both ways. Either we are strict on each other or wikipedia is garbage. Take your pick.
(And to the person who claimed anon IP’s cant make edits, you dont know what you’re talking about)
posted by tch
on September 5, 2006 #
Interesting, but as you comment, the analysis really needs to be expanded beyond a couple articles. Otherwise, you risk generalizing from too small a base.
Speaking of which, I’ll generalize - or at least talk about - myself. The first big thing I did on Wikipedia was editing the Henry James entry up to featured-article status, plus writing a lot of articles on individual works by James. I did much of this work anonymously because I was often too lazy to log in under my account name.
You could say I was your narrow-focus “expert” user at first, the kind of editor you think contributes most of the real content. But then something funny happened. I gradually turned into one of Jimbo’s 0.7% users. Now I make lots of edits to various articles on the encyclopedia, and almost none of them have anything to do with Henry James.
Yesterday, for instance, I worked on an article called Human Waste Project (don’t ask). Yes, I chip in some real content to such articles, but I also do a lot of cleaning up, linking, organizing, citing, etc. As a result, my edit count has started to pile up, and my contributions list has become a hodgepodge of edits on wildly different subjects.
So your two types of editors are not always distinct. Sometimes, a narrow-focus expert can turn into an all-purpose janitor. God help me, it happened to my little old self.
posted by Casey Abell
on September 5, 2006 #
Casey: Seth Anthony’s study of admin habits finds exactly what you describe: admins start out doing serious contributions, then become janitors. (He even uses the same term!)
And then, apparently, they burn out and leave (for a while, at least).
posted by Aaron Swartz
on September 5, 2006 #
Welcome to the world of the enlightened.
You’ve (re)discovered that most editors don’t edit most pages most of the time, via a different approach from the one Greg Maxwell used.
Thank you very much!
Now if only more people with clue were to actually help me tidy wikipedia guidelines, that’d be nice. :-)
posted by Kim Bruning
on September 5, 2006 #
Oh yeah, I’m aware of the pattern. I think Angela Beesley said something similar in an interview somewhere. One caveat: “janitor” may be something of a misnomer. To take another example, I recently “janitored” the article on John Podhoretz. I did a lot of reorganization, straightened out the cites, set up a bibliography, etc.
But I also added a lot of content about Podhoretz’ views on various issues. So it’s not like janitors don’t make content contributions, as well. We shouldn’t get carried away with the distinction between the two types of editors. They really do blend into each other sometimes. For instance, they’ve blended into, well, me.
posted by Casey Abell
on September 5, 2006 #
I can’t vote for you — I didn’t make 400 edits, so I don’t count. :-)
<
p>
Dunno if that confirms your suspicion. I can say I contributed substantial text to about 5 articles, 2 of which I just checked, and atleast part of my text is present in the article today.
<
p>
posted by agrajag
on September 5, 2006 #
You discovered some fascinating numbers and trends, how you present it a bit as a wall of text: Add some charts in there and you’ll have an internet sensation of an article.
posted by Dennis Forbes
on September 5, 2006 #
I don’t think there’s anything you can do to attract occasional contributors to the inner circle of Wikipedia - they just don’t care. They’re happy that they can share some knowledge with the world, and I highly doubt that Wikipedia’s policies lie in their area of interest.
posted by Baczek, an "occasional contributor"
on September 5, 2006 #
Your finding pretty much agree with what I’ve found from working on Wikipedia for a year or so. I’m a little surprised that it’s skewed quite as far as you say - I think most newbies start by fixing small problems - then get the confidence in the process to dump their personal knowledge areas into a couple of articles which they proceed to maintain for a while.
Then comes the fork in the road - some get drawn inexorably into the community and make more and more (mostly minor) edits - where others lose interest and go away.
But without doubt, it is impossible for 500 people to have made even the smallest scratch into 1.2 million English language articles. Over the couple of years that Wiki has really taken off, those 500 people would each have had to have written one new article a day in order to have written half of the articles out there.
When you consider the increasing amounts of time those people spend arguing about policy, voting on articles to promote and delete and fixing up style/category/spelling/grammar issues too - that’s just not credible.
Good article - I agree with you.
posted by Steve Baker
on September 5, 2006 #
Excellent article!
I agree with most of the things you said.
PS. If I could, I’d vote for you :)
posted by Luke Zapart
on September 5, 2006 #
This is an interesting analysis.
This would be an absolutely exellent analysis, if someone would whip up one or two graphs that depict the contrasting metrics, and demonstrate what’s going on.
posted by algal
on September 5, 2006 #
Well I’d also vote for you if i could but im not playing whack-an-edit with these 500 subeditors to do that.
I’d question the validity of any editor whom could make 500 changes, what do they know apart from say writing good english.
posted by Bananasin the Falklands
on September 5, 2006 #
Unfortunately, it does not work as well as we would hope. Check out the article in The New Yorker…
http://www.newyorker.com/printables/fact/060731fa_fact
posted by dave
on September 5, 2006 #
I can tell you from first hand experience that you are correct. What MADE wikipedia great (for the casual contributor) is the ability to “put there two
cents in”.Yes I think the contributors are far more important.The fact is that the editors try to run a dictatorship and delete most of the articles because
of format,I thought that was why they are editors in the first place. Now you have to log in to contribute in which about half as people will get involved. So now you have “run of the mill” encyclopedia not an information machine
way to go inovation killers
posted by just2speek
on September 5, 2006 #
Nope, you don’t have to log in to contribute. I still let a few anon edits slip in occasionally because I forget to log in.
Anyway, I’m pretty exo, which in wikitalk means I concentrate on articles and related content like images, templates and article talk pages. Very interestingly, my edit count profile looks an awful like Aaron’s. (Run Essjay’s edit counter and you’ll see what I mean.)
I like to follow some of the policy debates, and I occasionally get dragged into an article-for-deletion hoo-hah. I also copy the house newspaper, the Signpost, into my user page and make stupid comments on the stories.
So yeah, I’ve gone a little past the fork in the road towards the meta side. But not too far. I still throw in lots of content.
posted by Casey Abell
on September 5, 2006 #
Very nice work Aaron!
I was wondering re “random selection” of pages for your tests, … if you randomly select from the entire corpus, versus from the 1000 most popular (on some metric, eg. google rank), versus the 1000 oldest, etc… how the results would vary. Hope you can find some resources to dig further into this.
posted by Dan Brickley
on September 5, 2006 #
Speaking of random and non-random article lists, there’s an interesting list at the Signpost site of the 100 most popular articles on Wikipedia. You’ll notice an obvious pattern. Hint: sex sells.
posted by Casey Abell
on September 5, 2006 #
It is unfortunate that the qualifications for franchise serve to disenfranchise the most substantive contributors to wikipedia. Since you are attempting to represent them, your campaign is rather handicapped. I wish you the best possible success.
posted by aminorex
on September 5, 2006 #
You got my vote, man.
I’d like to see more numbers and charts and more on your methods. Heck, I’d like to rerun your experiment in HuWiki.
posted by nyenyec
on September 5, 2006 #
What sort of computing power would be required to get this run over a much larger (10-100%) chunk of the wikipedia?
posted by Jeremy Zawodny
on September 6, 2006 #
Wikipedia is past the first phase of its growth, when the emphasis was on seeding content. It has reached sufficient size that parasitic elements start to become noticeable. The emphasis must now be on cultivating an organization of editors that can properly tend to its growth. This will require creating a hierarchy of editors and creating a system so that this hierarchy can be properly created and regulated. For example, an easy complaint mechanism would allow a casual contributor like one of the commenters above to report his problem with an editor wiping out his content to another editor with wider authority, who can adjudicate. Further, he might be given the ability to create a petition making a case for removing an administrator from control of a particular section. This petition might be enforced if he can get enough registered users to vote for it. Those who believe that anything can be done without some sort of administrator hierarchy and easy ways to collect consensus opinions are hoping for a utopian fantasy. Wikipedia may be a good example of the tragedy of the commons, as most contributors, including me, cannot be bothered to look into the administrator selection process. However, Wikipedians must always be aware that someone can fork the content and create their own system with its own editing hierarchy (though the costs of bandwidth and servers will ensure that this is not a spontaneous decision). I suspect that forks to other sites with better administrator processes is what will happen.
posted by Ajay
on September 6, 2006 #
No successful forks have happened yet. Fred Bauder, one of the ArbCom members on Wikipedia, tried a fork with Wikinfo, but it never really caught on.
Anybody who tries to impose a hierarchy of editors on a Wikipedia fork will find it hard to lure contributors away from more free-wheeling sites, like Wikipedia itself. That’s what happened to Bauder, and I suspect it will happen to anybody attempting a more rigidly policed fork site.
posted by Casey Abell
on September 6, 2006 #
Aaron,
Excellent article, and congratulations on your slashdotting. As per your last article I had gone to wikipedia to vote for you and found out that because I had not made more that 400 edits I couldn’t vote. I was surprised that I didn’t meet the criteria as I’ve been a member for a long time. I decided I must not be a very good member so I doubled (by doing ten) my number of edits instead of voting.
Now reading these points I am all the more motivated to vote for you but have decided that instead of me being a poor wikicitizen wikipedia’s voting system advances an unfair assumption. The same assumption that you unearthed here: that the only people that matter are the large quantity editors.
So I can’t vote for you, but if you do get elected and you do believe what you wrote you should act to change the voting system.
-Andy
posted by Andy Schilling
on September 6, 2006 #
You seem to have forgotten considering those users with large numbers of edits and who contribute large amounts to articles. There may be those who are not by your definition “outsiders”. What are the statistics in this instance? This would have made your analysis a little more complete.
posted by
on September 6, 2006 #
Intreguing study, Aaron. From my own personal (& incomplete) experience, I have to agree with your conclusion — either I can focus on lots of little edits that concern style, formatting, & so forth, or I can focus on making a few substantial contributions.
Despite having written a number of essays in college, I still find it a challenge to write even 500 words on any given topic for Wikipedia.
So why do some contributors cluster at one extreme or the other in their behavior — & some end up becoming entirely involved in another axis of participation, creating & enforcing policy? I sense material for a Master’s thesis in psychology or sociology here.
posted by llywrch
on September 6, 2006 #
A nice article and thx a lot for the insight!
Anyhow i do have some concerns:
One interesting thing about the wiki-thing in wikipedia is that different tasks (like editing, reviewing, formal qa - to name but a few) are conducted through a single interface and that the complexity of creating an article is therefore driven from the software itself into the flexible organisation of production and publication (so to say, the human factor).
Regarding yours and Jimmy Wales research, i think that both approaches show some flaws in terms of this aspect.
You both wanted to identify an aspect of quality - Which contributions by whom made up the “essence” (a hard to operationalize term in anyway) of the article by quantitative means. I don’t know if this works at all, but in any case there needs to be some sort of categorization:
What kind of edit has been conducted here? Editing, Reviewing, Janitoring (?) etc.
Did this edit lead towards the current state of the article or not? The so called “quality” of an article in wikipedia is not defined by absolute means but is the result of the quality demands of the stakeholders, therefore subject to dynamic change.
In the end this approach is a tedious and time-consuming work if you want to conduct it for thousands of articles.
And also, by the way, not as shiny and reliable as pure quantitative work.
posted by Lars
on September 6, 2006 #
“admins start out doing serious contributions, then become janitors.”
<
p>Close. I sort of started on both, got way deep into en:wp administration, then into Foundation stuff, and now I spend my time on content (both copyediting tweaks and writing new short articles) and Foundation.
<
p>I no longer even look at my watchlist. They’re not my articles.
posted by David Gerard
on September 6, 2006 #
Very interesting article, and definitely worth reading. I often feel that the ‘Wikipedia community’, such as it is, places too much emphasis on users with a large number of edits (i.e., themselves), and doesn’t consider what those edits actually are. The current voting system is a perfect example of this. I would be voting for you, Aaron, but apparently I have only 85 edits. What I don’t understand is why this makes me less deserving of a say in how Wikipedia is run than someone with more edits.
posted by Alasdair
on September 6, 2006 #
I’m also one of the ‘hybrid’ users mentioned above. When I first came aboard 2 1/2 years ago, I went crazy adding content on local institutions and issues, plus topics I know about but didn’t have significant content.
But I only know so much; after a while I was mainly just maintaining what I’d contributed and talking about how else to present it, with other editors who were interested. Once in a while I’ll have a burst of new knowledge - I’ve visited a location or read an article or experienced a unique event - where I can dump a bunch more knowledge. But those are few and far between; 90% of my edits these days are cleanups, formats or comments, of both my work and of existing work that I have interest in and knowledge of.
I think many ‘diehards’ are the same; working to keep things looking good and presentable and encyclopedic to the outside world.
I can’t speak for the ‘knowledgeable newbie’ crowd - I don’t know them well - but we should keep them in mind when making policy decisions. We try to assume good faith and not bite them, as the policy pages say; how realistically they are followed is another matter. That, we will have to leave for the administrators.
I’m still considering my vote. Aaron, you may well be on my final list; I’ll cosider all the issues when it comes to it.
posted by Radagast
on September 7, 2006 #
Thanks for doing this analysis, it’s very interesting.
I agree with algal that a graph would be good. I tend to be text-oriented myself, but I know that a lot of people skim the text but look at the figures when reading research.
posted by Bayle Shanks
on September 7, 2006 #
Aaron, please provide Jeremy Zawodny with any guidance you can on the work involved. He has the resources to get the work done.
posted by
on September 7, 2006 #
Interesting article, I agree with your findings that articles are written mostly by anonymous (expert)contributors.
Mostly they EXPAND the already existing (small) article. I think article creation is less common among anonymous contributors (this would be interesting to find out as well, or maybe there are already statistics on this??).
The value of CATEGORISATION and creating INTERLINKAGES I think should be stressed more in your article, since these improve the encyclopedia and create new insights.
categorisation/interlinking effort are probably more often undertaken by the active community (does statistical data exist?).
Another point I’d like to make in my concern with DELETION of whole articles on wikipedia by often the active community: after a small week and some votes of this community on a certain article, the article may be voted to be deleted. This is often decided quantitatively, not qualitatively. Often arguments for keeping the article are not debated; or arguments for deletion are not well founded.
Some contributors might be on a wikibreak, come back some time later and find out (or not, even worse) about a deleted article: old deleted pages are (sometimes?) completely inaccessible.
This is a loss of content that should be avoided since it frustrates contributors and decreases the quality of the encyclopedia for its users.
Feedback on these concerns is much appreciated! :)
posted by Whirl
on September 8, 2006 #
Good article, interesting findings (thanks for the numbers …). I’ll publish a little summary on my blog poiting to your article.
posted by PageTurner.info
on September 9, 2006 #
Great article, clear and interesting!
posted by matt
on September 9, 2006 #
Contributeur français régulier, je peux confirmer que mon activité se partage entre quelques contributions substancielles dans mes domaines de spécialités et des interventions “d’éditeur” (forme, corrections, vérifications, critiques) beaucoup plus nombreuses. La loi de Pareto s’applique mais dans le sens 80 % des contributeurs pour le contenu et 20 % pour la forme.
L’intérêt de Wikipedia n’est pas dans sa ressemblance plus ou moins proche avec les encyclopédies traditionnelles (recueil de l’information, rédaction, validation) qui est inévitable, je ne vois pas comment faire autrement !, mais dans le fait que l’ensemble de ces actions est ouvert à un grand nombre de personnes (potentiellement gigantesque). Car désormais les effets produits sont des régularités, des régulations qui dépassent celles qui étaient jusqu’alors accessibles : le recueil des informations sollicite des dizaines de milliers de personnes et progressivement de plus en plus, ce qui permet de couvrir des domaines qui n’étaient jamais couverts par les encyclopédies traditionnelles, limitées par les ressources de temps et matérielles. Les corrections-validations par des milliers de lecteurs produisent au fil du temps des articles de qualité acceptable qui ouvrent sur un thème et donne les moyens d’élargir la recherche et d’en faire éventuellement la critique (combien de fois suivant un lien externe à WP, j’ai pu critiquer une information et en proposer ou faire la correction dans un article ?). Ce processus est cumulatif dans le temps et accroît encore l’effet “grand nombre” : ne pas en tenir compte serait une grave erreur d’analyse. C’est d’ailleurs une erreur que certains expriment en demandant de manière récurrente des validations par des experts ou des comités restreints : une manière de supprimer l’effet “grand nombre” qui me semble-t-il est la force majeure du processus Wikipedia. Cette force en évolution permanente fait peur à certains, manifestement.
posted by Jean
on September 11, 2006 #
This article will have a profound impact on how I approach my own wiki project — http://toastmasters.wikia.com.
I’m one of two “super” editors who have made the bulk of edits on the site; I’m now “out of content” — i have, literally, not much more to add.
I realise now that I should be approaching so-called Subject Matter Experts and getting them to add content, and letting the rest of us tidy it up for them.
Thanks for this insight — you have altered policy on one wiki at least!
posted by Erich Viedge
on September 11, 2006 #
Great proactive analysis. Wales’ statement, prior to his “next version”, probably shouldn’t have been so definite..
Last month Jason Calacanis wrote, about the pitfalls of Wikipedia’s syntax and unintuitive interface. Given your findings and as a candidate for the foundation’s Board of Directors do you feel a leveling of the learning curve of the content production process should be of priority? After all, if the content producers are empowered to improve their formatting and structure, the “Top 2%” will have less ground to accumulate edits and thus power.
posted by Angelo Gladding
on September 13, 2006 #
Thanks, maybe one of the best articles i ever read about wikipedia.
Thank you very much for this result.
From my point of view this is confirm to out assumption that there must be a kind of critical mass, after that a wiki system will work.
Maybe also from interest our research paper:
http://elearningblog.tugraz.at/archives/107
posted by Martin Ebner
on September 14, 2006 #
I’ll have to agree with you here. I know that I’ve only made a few contributions to Wikipedia, mostly on subjects that I know enough about to make meaningful contributions. So occasionally I’ll jump on Wikipedia and make a few edits, usually substantial in content (in accordance with the “be bold” imperative of Wikipedia).
Although it hasn’t always happened, largely I find that the pages are monitored by a specific person who end up attempting to erase the edits or substantially change them. Take a look at the Wikipedia page on Jack Thompson, which is basically edited into a hagiography. One of the pages I made a substantial edits to (roughly doubling the length of the article) was then edited back to its initial state by a person with a severe conflict of interest (vice-president of a company who was mentioned in the edits). What it often comes down to is that the people who are compulsively looking out for their own interests, rather than the truth, are willing to dedicate more time on the site squashing what they don’t like. Too bad, really.
posted by Anonymous Contributor
on September 14, 2006 #
I wrote an ethnography of a Wikipedia article a bit more than a year ago, looking at the Support Vector Machines article.
Though my conclusions did not directly contradict those above, I did not find that the majority of major edits were made by less active users of Wikipedia. For the Support Vector Machines article, I found that there were two “major editors” - one of whom was a very active Wikipedia user with an account and second was only known by an IP address but knew the norms of Wikipedia very well, indicating he/she too was an active Wikipedia user.
posted by Caroline Moore-Kochlacs
on September 15, 2006 #
Wish I’d seen into this essay earlier. This is something I’ve believed for a while, but it’s nice to see someone confirm this by research.
posted by James Yolkowski
on September 17, 2006 #
Absolutely fascinating analysis. Well done. Thanks for posting.
posted by Mahdi Gad
on September 27, 2006 #
You purchased time on a cluster to analyse “a few articles”! The analysis seems basically correct, however look at people like Charles Matthews and Mav - they make both substantial and substantive contributions.
posted by Rich Farmbrough
on October 13, 2006 #
I wish there were more of these analytical studies like your or the famous correspondance on Nature. Wikipedia is becoming too much of an important phenomenon for our society to let people speculate on how it works:we need to know as precisely as possible how it actually works, what kind of impact has on people and how can be improved.
posted by Giorgio
on October 24, 2006 #
Respected Sir:
I am a student of class X and wants to complete the project work on numbers 0,1,2,3,4,5,6,7,8 & 9 discovered by whom in ancient times. In fact a writeup is required to be submitted to the Ryan International School, Faridabad.
Kindly help urgently as I am unable to find the site in wikipedia.
An urgent response on above e mail of my father will help me
Thanks & regards,
Vinayak Takru
Class X
posted by Vinayak
on October 26, 2006 #
i’m not sure, but shouldn’t there be ‘140’ next to the ‘2%’ instead of ‘1400’ in the third paragraph?
posted by jnst
on November 16, 2006 #
I think that the regulars play a role that is larger than ‘formatting’. Part of the reason that Wikipedia is not a useless mess is because it is organized. The same collection of information, poorly organized, would be little better than doing a Google search. For many people, in many situations, the current Wikipedia is a more useful way to find certain kinds of information than a Google search is.
Your observation that there are different types of users is spot on, but I think that all types (except vandals) play an indispensible role.
posted by Rafael Garcia
on November 16, 2006 #
I’d wager that the statistics are wholly made up and no such code exists. I’m no proponent of Wikipedia, but I call bullshit on the article. From what I’ve read, I believe that Swartz writes fiction.
If what he has written is true, it would be easy for him to prove me and many a skeptic wrong, but I believe he cannot.
JD
posted by J Delphi
on December 18, 2006 #
That’s a strong and baseless charge, J Delphi. I’ve provided the code and details on request to those who have asked, other studies have replicated the results, and a larger study is underway.
posted by Aaron Swartz
on December 21, 2006 #
I have been a “original content generator” on Wikipedia for some time, but stopped after I’ve become disillusioned with the whole site. I do not see very many users adding original content at all (at least on the articles I’ve worked on), other than just the moving, deleting and rewording things here and there. What we have are the some smuggest, know-it-all editors and admins making snide remarks on the work I have done, but they haven’t done anything to add anything worthwhile. But that’s my experience.
posted by Calvin
on December 29, 2006 #
That’s the experience of many people who have tried to participate on Wikipedia, Calvin:
http://nonbovine-ruminations.blogspot.com/2006/10/sigh-wikipedia-needs-better-admins.html
http://mail.wikimedia.org/pipermail/wikien-l/2006-October/054949.html
http://www.cow.net/transcript.txt
http://www.americanchronicle.com/articles/viewArticle.asp?articleID=11109
http://www.roughtype.com/archives/2006/02/community_and_h.php
http://www.techybytes.com/is-technology-slowing-down-freedom-of-speech/
http://catallarchy.net/blog/archives/2006/12/20/the-stupidity-of-crowds/
http://www.andycarvin.com/archives/2006/03/wikipedia_blocks_sch.html
http://wikipediareview.com/index.php?showtopic=4
http://www.aaronsw.com/weblog/whowritescomments
http://pilotguy.wordpress.com/2006/12/20/where-have-all-the-admins-gone/
http://thechrisd.spaces.live.com/Blog/cns!90D096458FBAE74E!242.entry
http://stabani.com/archives/2006/247
http://www.techcrunch.com/2007/01/01/wikipedia-bans-qatar/
http://news.bbc.co.uk/2/hi/technology/6224677.stm
posted by Doug Holton
on January 5, 2007 #
Most of my contributions have been removed (not modified) and so I have largely stopped contributing.
Often times the contributions are removed and not modified with obscure references to numerous wikipedia policies that seem to make no sense and that are enforced very unevenly across the wikipedia.
At other times, controversial but accurate information is just removed, resulting in a very bland innocuous mostly harmless article. (And not terribly useful.)
The cyberstalking article is a case in point. It is humorous (and sad!) to examine how that article has evolved over time.
posted by Jerry Asher
on January 30, 2007 #
Aaron,
I confess. I spend entirely too much time patrolling the web for reading material. But I had to pause to pile on here: this piece in that rare “splendidly good” category. Thanks for the good read.
I’m sorry you didn’t win the election, but, ironically, it’s rather obvious why, eh?
posted by Bill
on February 7, 2007 #
What annoys me about Wikipedia is the people’s penchant for rewording other people’s stuff. For instance, I wrote original material on “Impression Fraud” on my site( http://www.sofizar.com/impression-fraud.php ). I was shocked to find that wikipedia entry for impression fraud was a summary of my article with nary a credit. Are the authors of wikipedia just trying to “get done” with everything?
posted by Zafar Khan
on February 12, 2007 #
Let’s face it. Few people want to strain themselves of doing the hard and winded research (such as going through books or scouring through newspapers and academic journals, and so on) and writing. It’s easier just to rephrase someone else’s work or use other Web sites as citable “sources” as accessible through Google.
posted by S.
on February 12, 2007 #
I wonder if all the people who have had trouble with “mean administrators” really provided valuable content. Most of them never name the articles they worked on so that you can’t check if their claims are true. Just having a point of view doesn’t necessarily mean that it’s reasonable oder widely recognized.
posted by Carbidfischer
on February 14, 2007 #
I don’t know. Just what is it with the current zombie-like fascinations with Wikipedia, MySpace, and so on?
Perhaps Wikipedia simply gives people a certain surge of authoritative and administrative power and prestige over others. Those with user names feel entitled to a sense of superiority, compared to a user with a mere IP address and a few original entries.
posted by A.T.
on February 20, 2007 #
Great article. One time, I intentionally added factually correct info but with some intentional spelling and grammatical mistakes on a not-so-popular article, but it hasn’t really been touched for nearly 9 months already (and still has not to this day!). All the article received from one editor was a “Citation needed” notice. So where are the lovely by the book editors?
posted by J. Jeffers
on February 21, 2007 #
Perhaps they have nothing else to do with their spare time.
posted by !
on February 26, 2007 #
“But on the other hand, the more people view an article the more it is likely to be corrected and balanced for NPOV.”
The more people that view an article the more likely the article is to reflect the views of more people. Sometimes that will tend towards a NPOV and sometimes not. The article will increasingly reflect popular opinion (which may not be neutral at all).
“it isn’t perfect, but in the opinion of many the results are fairly acceptable.”
Perhaps, but not definitive. How do you justify that claim?
When an article can be objective without threatening the views of people it stands a decent chance of being so. Wikipedia is, by its nature, not an objective resource. It is a useful one, though.
posted by Iman Darwiche
on March 9, 2007 #
I think the usual problem with Wikipedia is that there is the expectation that someone else will do the work.
posted by Lian Wen-chiu
on March 22, 2007 #
I am agree with autors opinion. I think the volunteer authors of Wikipedia articles don’t have to be experts, although some certainly are. They could be anyone! Further, volunteers do not need any formal training before creating a new article or editing an existing article. Many people have created or edited articles in Wikipedia. They live in different countries of earth, from all ages and backgrounds. The autors are called a “Wikipedian” or “Wiki.” The policy to add to the encyclopedia only statements that are verifiable, and not to add original research. To hold the quality high there are special style guide encourages editors to cite sources. I love WIKI
posted by John Miller
on April 17, 2007 #
Most of my contributions have been removed (not modified) and so I have largely stopped contributing.
posted by Maca
on May 3, 2007 #
Absolutely great analysis and the follow ups on this were also great.
posted by Amit Dixit
on May 8, 2007 #
This article is vital; I’ll be sharing it with everyone I know…I only wish I’d found it sooner! It is so reassuring to find that someone else understands — someone with the expertise to illustrate the problems of Wikipedia in a way that cannot be swept aside by the “insiders.” I often find that my own lack of Wiki-protocol knowledge puts me at a disadvantage when arguing points on the talk pages. Thank goodness for WP:IAR (my ace in the hole)! The atmosphere at Wikipedia has become downright draconian; thanks for empowering me to exhale….
posted by Discouraged Editor
on May 10, 2007 #
Very good article Aaron. I had this suspicion for a long time myself. Just couldn’t pursue it for various reasons. I am not a particularly big fan of Wikipedia or Jimbo Wales due to their very weird policies of editing. To date I have contributed only one big article and that thankfully survived massive editing. I feel those folks should open up a bit more. It seems way too rigid at the moment. Vandalism is inevitable but the community should grow. Stagnation will not only erode the quality of the site but also lead to its slow and painful death. I hope you read this Jimbo.
posted by Timmy Jose
on May 16, 2007 #
Very good article Aaron. I had this suspicion for a long time myself. Just couldn’t pursue it for various reasons. I am not a particularly big fan of Wikipedia or Jimbo Wales due to their very weird policies of editing. To date I have contributed only one big article and that thankfully survived massive editing. I feel those folks should open up a bit more. It seems way too rigid at the moment. Vandalism is inevitable but the community should grow. Stagnation will not only erode the quality of the site but also lead to its slow and painful death. I hope you read this Jimbo.
posted by Timmy Jose
on May 16, 2007 #
It’s not the first time in the history of mankind that a Great Thing is taken over by “admins” who don’t understand much of the Initial Idea, but who understand very well how to benefit of it. Hierarchies, structures and rules prevent any further progress and the Great Thing converts into a Serious Thing, that uses the Initial Idea only as a marketing tool.
posted by Severin
on May 17, 2007 #
In response to the criticism made by “Carbidfischer”, well, if you have something to benefit from Wikipedia I’m sure you’ll defend them.
Anyhow, I hope the author of the original article could do some similar analyses on the Wikipedia articles on Chinatown, Asian supermarket, and Little Saigon. You see, I wrote a bulk of those articles and over the years I have seen many others just delete here, rephrase there, add a link there, make remarks thither but not much else.
posted by Frustrated one
on June 7, 2007 #
It is true that the Editors,or the formatters,are more concerned about the tone,style,category of the wikis.While most part of the world does not find any or even minimum adequate representation,(Even historic monuments and large towns in India do not appear even in the www,what to say of the wikipedia)the concern with formalities does discourage contributions.Contributors do so because they felt something about the topic and the feeling will certainly come out.All of them,or even a few of them cannot be expected to have the attitude and training of encyclopedia editors.The editors appear to be specialists who know only editing and surely they seem to be knowing nothing else.In India,the office typists have the same attitude.They show all the concern for the format only,not for the content,because they do not realise the relative importance of the content.
posted by N.Krishnamurthy
on June 9, 2007 #
Dear Aaron,
why do you delete my comment!
You wrote in your article: “Thus his assumption that outsiders contribute mostly vandalism and nonsense. And thus the comments you sometimes hear that making it hard to edit the site might be a good thing.”
If �outsiders� like me give a different viewpoint you think the same way like Jimbo Wales.
Many Regards
Roland
posted by Roland
on August 5, 2007 #
Great post Aaron. I’m a CS grad student at Stanford and the sad truth is that the investigation you did as part of your blog post here is far more worthwhile than 99% of the so-called “research” and paper writing that happens in the CS department here.
posted by Jeff Malkes
on January 16, 2008 #
Wow, great article. Im glad to hear your findings, after hearing Wales initial findings. It has far reaching implications in society as a whole, not just Wikipedia. How depressing would that have been, that only some .7% of people were capable of any kind of productive contribution to society, with the rest of us just sitting around bickering about the presentation. Though it may seem like that to those with some kind of deep interest and active role in society, really all they are doing is directing everyone elses major contributions. In spite of their self deception to the contrary, their roles as spit shiners is immensely important, and not at all petty. If they could get off of their imaginary high horses to see that, then they probably wouldn’t need the delusion to keep their self image so well bolstered.
posted by Jesse A. Rudolph
on August 30, 2008 #
Great article, I find some of the people policing wikipedia near stalinist. And often not knowledgable to an astonishing degree.
posted by ralf
on October 14, 2008 #
Thanks for the provocative and interesting analysis. I don’t agree that the editors of traditional encyclopedias concern themselves largely with formatting and presentation. As mediators between the copyeditor and the contributor, their primary objective is to vet content: to assign articles, assure the reader that the contributors have some grounding in an existing literature to establish the authority of their claims, to check facts, and especially to align articles on various subjects with other articles: to remain true to the encyclopedia’s overarching purpose. The insider-editing you describe in your article is largely basic copyediting, but it is far, far narrower than “traditional” encyclopedia editing, and it’s a significant reason why academics and instructors are wary of Wikipedia as an authoritative source as opposed to a window into the minds and ideas of contributors.
posted by Peter Knupfer
on January 3, 2009 #
You can also send comments by email.
Comments
This is interesting. I came to a similar conclusion (that Wikipedia is similar to traditional encyclopedias) based on my own experiences as a contributor of traditional encyclopedia articles; see here: http://www.stat.columbia.edu/~cook/movabletype/archives/2006/08/wikipedia_and_e.html
Also see here for my further thoughts on the topic: http://www.stat.columbia.edu/~cook/movabletype/archives/2006/09/who_writes_wiki.html Regarding the formatting issues that you mention near the end of your article, I agree that ultimately the ocntent is more important. But I imagine the common format is a big part of Wikipedia’s appeal—it sort of makes it into the McDonald’s of information sources. I suspect that the clean and uniform format is a large part of Wikipedia’s air of authority.
posted by Andrew Gelman on September 4, 2006 #
Excellent article.
posted by Paul Simpson on September 4, 2006 #
An excellent article. I tried to vote, but since I am one of your “occasional contributors” (I’ve edited only one article to make content changes), I am not eligible to vote. It appears that the opinions of “occasional contributors” will not be heard.
posted by Eric on September 4, 2006 #
Great article, Aaron. Will you release your code and conclusions?
posted by Cory Doctorow on September 4, 2006 #
I need to clean the code up before I can publish it, but I’m happy to work with anyone who’s interested in pursuing this further.
posted by Aaron Swartz on September 4, 2006 #
Andrew Gelman says “I suspect that the clean and uniform format is a large part of Wikipedia’s air of authority.”
Spot on, BUT the “air of authority” is quite a different matter than actual “authority”.
The good sign is that format, etc. sells tickets but content changes the world.
The bad sign is that those who groom this monkey seem to have taken control of the project by setting themselves up as a hierarchy via voting eligibility rules.
The people Aaron shows to have crafted the content are largely disenfranchised while the “eyeshade editors” rule.
Hope I’m wrong.
Love.
posted by William Loughborough on September 4, 2006 #
“Don’t worry, he tells the world, Wikipedia isn’t as shocking as you think. In fact, it’s just like any other project: …”
And he’s correct, though that cuts both ways. There’s an “editorial staff” which handles the copyediting, versus the occasional freelance contributors, who work on specific topics. That is indeed a standard model.
What’s really interesting about Wikipedia is how:
1) It manages NOT TO PAY almost anyone, getting people to donate their services, on the basis of “community” or other ideas.
2) It’s developed elaborate intellectual defenses against the failings of #1, whenever the flaws show through.
Those are two aspects which are, if not exactly completely novel, definitely innovative implementations of eternal themes.
Let’s again recall there’s $4 million of venture capital invested in the associated Wikia venture, and at a rule of thumb of 10x return, the investors want at least $40 million. That’s got to come from somewhere, and figuring out how to get work for free seems like a major part of it.
posted by Seth Finkelstein on September 4, 2006 #
Seth, you seem to think those venture capitalists are being rational in expecting Wikia to succeed to the tune of $40 mill; I’d say that that is rather optimistic and reminds me not a little of the first Internet bubble.
posted by maru on September 4, 2006 #
You are obviously speaking of good encyclopedias. I had a wild boss who, in his earlier years, got an editorial job with one of those grocery store encyclopedias. He specialized in R through S.
Julie
posted by julie on September 4, 2006 #
“Observe: The humorless wikipedia administrator in its natural habitat!” - me, looking at the ‘recent changes’ page of the Encyclopedia Dramatica
There are the people who contribute content, and there are the people who maintain it. The former doesn’t know the intimate details of the wiki’s features. The latter does, and will sometimes leave snippy comments about the lack of this as they make it conform to the spec.
The mindset is the same, whatever wikipedia-type place it’s found on. Even in ones dedicated to misinformation and gossip.
posted by Egypt Urnash on September 4, 2006 #
Fantastic insight.
posted by The Liberal Avenger on September 4, 2006 #
Very insightful Aaron.
In fact to think of it, I guess it is this flawed understanding of the nature of the contribution process to Wikipedia which Wales harbors might be behind the recent decision by Wikipedia to ban edits by users who are not logged in.
I think this is one of the most regressive moves in the history of Wikipedia - by putting in a login ‘barrier’ i would assume a lot of the casual (but as you pointed out, very critical) contribution is put off.
This type of casual contribution is at the heart of Wikipedia, and no amount of vandalism is reason enough to put barriers around this. It is like killing your goose which lays the golden eggs, just because you cant stop some of them from being stolen.
posted by neeraj on September 4, 2006 #
I already voted, and now I’m annoyed I didn’t vote for you!
<
p>This is a fantastic observation. If you weren’t running in the Board election, I’d be spreading it far and wide. As is, I feel too far into the guts of Wikimedia politics to be spreading it with propriety and not appearing to push your candidacy per se … gah.
<
p>But yes. This has major implications for policy and how Wikipedia works.
<
p>(And it’s also a nice big stick for me to hit people over the head with concerning Byzantine overengineered processes.)
posted by David Gerard on September 4, 2006 #
David,
You can vote again, and it will override your previous vote.
posted by Jacob Rus on September 4, 2006 #
What computer cluster did you use? What kind of programming environment is it?
posted by Will Bendick on September 4, 2006 #
Aaron - this page was noted on wikien-l, and I suggested to Gregory Maxwell that you two should talk about how to get good data across the whole database. Greg is very good at running interesting queries on the whole database - if your methodology can be automated, this will give you accurate data. Since if you’re running for the Board because of this, I assume you’d prefer greater certainty!
Thread starts here: http://mail.wikipedia.org/pipermail/wikien-l/2006-September/thread.html#53307
posted by David Gerard on September 4, 2006 #
i love wikipedia.
but i have felt that my contributions — on topics where i do have expertise — have been warped by people who merely want to “make it sound like an encyclopedia” without having any knowledge of the topic.
it seems to me that the experts that wikipedia so desperately needs will be precisely those people who are not making edits to a large number of topics.
-bowerbird
posted by bowerbird on September 4, 2006 #
Interesting article, but I think you’re being overly dismissive of the insiders/formatters. Because the likelihood of casual users making significant contributions is determined not only by how easy it is for casual readers to contribute, but also by how many such readers there are to begin with. And the size of that pool of readers depends to a great extent on how readable, reliable and accessible the information presented is. So a change that imposes a minor impediment to casual edits can still result in a net increase in casual editors if it also helps the “insiders and formatters” improve Wikipedia’s usability, and increase its readership.
posted by Abou Ben Adhem on September 4, 2006 #
I second what bowerbird says. I used to be one of the people making small but regular contributions to Wikipedia, on the subject of places I have lived and the London theatre scene/history, both subjects on which I have a lot to contribute, but got fed up of the self-appointed officious jobsworths who would come along minutes later and revert your contribution or substantially modify it (so that it lost the important detail), in order that things would fit “their vision” for the sections they “patrol” and have “made their own”. My time is too valuable to argue with these people, sit monitoring discussion pages, or fight for ages to wrestle control back (and in doing so become as bad as these people, if I were to end up spending my entire waking hours on Wikipedia as they appear to).
I’ve also become annoyed with the behaviour of this perceived “inner circle” when it comes to current events and the “high and mighty” attitude they have to outsider contributors (not me in these cases).
Wikipedia was good up until 12-18 months ago but lately it’s become far too cliquey. Which scares a lot of knowledgeable would-be contributors off, because if we are to invest our valuable time contributing some expert knowledge on some subject, we want to know that our work will remain there for others, and not just keep getting reverted out in seconds by some control freak that knows nothing about the actual subject.
Why am I taking the time to make this point here? Because your article proves the existance of this “inner gang” that I feel are actually holding Wikipedia back. To allow Wikipedia to grow and really pick the brains of the experts around the world, you need to do something to break up this inner gang and the mini empires they are building for themselves.
posted by Ian on September 4, 2006 #
As an “occasional contributer” I’ve found largely the same thing. The “super editors” are the ones whom the site benefits, and, unfortunately, they share the same opinion as Wales that they are responsible for the bulk of the content. I’ve proposed the addition or deletion of a couple articles, cited Wikipedia policy (two or three points in each case) and have been told point blank that I am wrong and that “the other editors would agree with” the editor I was arguing with. I may not spend 10 hours a day on the site or have thousands of edits under my belt, but this sort of treatment just left me with a bad taste in my mouth, and it sounds like the founder is all but condoning it.
posted by Anonymous on September 4, 2006 #
All of the original content before it was switched to PHPWiki was done by an almost entirely different group than the people who are doing it today, so while there may be 500 active supercontributors at any one time, between 2001 and now there has been a constantly changing group of active contributors. This has an effect on both the numbers that are being used here, I think. Early on, there was only about 500 people, total contributing, but everyone was a supercontributor.
As a side note, it was much easier to contribute before, when you could discuss with other contributors and know that someone really had to be disruptive to be blocked. It happens way too easily now, primarily I think because of a number of truly unprofessional so-called “moderators”. The power imbalance now is very anti-wiki.
alan_dershowitz (one of the earlier contributors.)
posted by alan_dershowitz on September 4, 2006 #
This is a smart and important post.
posted by Petey on September 4, 2006 #
Dear Aaron
As another occasional contributor I’ve established 2-3 articles and watched them get a life of their own.
When I tried to vote there was a curious response:
Good luck with the election.
Simon
posted by Simon Pockley on September 5, 2006 #
Regardless of how many people contribute to Wikipedia content, I remain concerned about the unchecked biases of contributors. Anyone can can make impressive-sounding citations to create an air of authority when writing/editing a topic. But what percentage of those occasional, unregistered contributors who are subject matter experts on that topic review new contributions to verify the changes? While this may be trivial for celebrity bios, it is particularly important for controversial scientific entries, such the standard model for physics, xenoestrogens, and global warming.
At least with a traditional encyclopedia publishing process, the compiling company hires people with well-established expertise to check the work of free-lance contributors. And on the most important topics, the compiling company gets top-notch subject matter writers to begin with. None of this is guaranteed in cyberspace.
As a test of my theory the problems with the lack of peer review in Wikipedia, I’ve augmented a topic (with citations) on a controversial medical topic to see if anyone bothers to challenge my assertions. It will be interesting to see the revisions of that article.
posted by jake3_14 on September 5, 2006 #
Great work, great post. I would vote for you if I was in the 400+ club.
posted by david mathers on September 5, 2006 #
Your point is well taken, with a caveat: that it’s difficult to actually quantify particular anonymous users’ contributions to Wikipedia.
As anonymous users are tracked by IP address, any one real-world user may be broken up into a countless Wikipedia ‘users’ if they happen to reside behind a dynamic IP.
Even given a fairly stable IP address here at home, the Wikipedia user contributions page for my latest anonymous ‘identity’ only lists 23 edits to my name. Another one from a month or two back lists only 6. My IP address at work seems to switch on an almost hourly basis, so multiple casual edits from even a single day from there may show up as isolated edits by random readers passing in the night.
The idea that the bulk of Wikipedia’s substance comes from the more heavily specialized, non-community users strikes me as inherently reasonable (and actually quite reassuring).
I don’t think, though, that you can establish with any certainty whether these anonymous users return time after time or if they really just make a handful of changes and never hit ‘edit’ again.
posted by Dan on September 5, 2006 #
This is fantastic research and very actionable. Unfortunately, as you note, some actions are going the other way (making it harder for the less frequent but more knowledgeable users).
The other major grip I have with Wikipedia recently is the proliferation of citations. I understand why people think they are of value but I feel like a) they lead to another false sense of authority and b) they lead to disjointed articles (ie., I’ve seen articles that seem like a long list of citations with not a single sentence seeming to flow from the preious one).
posted by pwb on September 5, 2006 #
I wrote several wikipedia articles, where I wrote an entire article up out of nothing. In each case, the article was then “edited” — without any notification to me — by some super-editors, who removed content, and turned what I thought was gosh-darn good writing into crap. I tried to explain to friends my frustrations, and they told me “Why do you even bother? Of course it’s just a bunch of self-important snobs.”
So I went and looked at the people who “edited” my contributions, and saw that my friends were right. These people, by and large, “edited” thousands of articles. In most cases, these edits were to remove material that they found unsuitable. Indeed, some of the people-history pages contained little “awards” that people gave each other — for removing content from Wikipedia.
Since I heard of your article, I headed on to the voting page, and, alas and alack, I am not eligible to vote, since I have only made 21 article edits. I was also informed that since I didn’t have the required number of edits by yesterday, I cannot become eligible.
HURRAH, I am DISENFRANCHISED.
All this leads me to ask, “I wonder if those little “awards” that people give each other count towards the required number of edits?
There’s an old saying, “The lower the pay, the higher the politics.” I’ve seen it happen at college student activities, non-profit volunteer community organizations, etc.
I believe it’s happening here, now, with Wikipedia.
posted by Bill Coderre on September 5, 2006 #
Perhaps a more useful metric would be: # of letters typed that are still present in the current release. This would weed out those who contribute garbage data that is only removed later. An even more useful but difficult metric to measure would be: # of letters typed that are present in the most recent non-garbage release (as the current version of some pages may be vandalized).
posted by Liam Morley on September 5, 2006 #
did anyone check how many of Wales’s top .7% of most contributing authors are actually bots?
posted by tarkowski on September 5, 2006 #
I think that this is very insightful and reveals the great ‘black hole’ that wikipedia could be drawn into. If I had the chance to vote, it would be for you. I fully support your campaign and would be interested to hear the outcome.
posted by SonicSpeed on September 5, 2006 #
I’ll join in saying: I’d vote for you, but I’m well under 400 official edits. I’ve made a lot more than my credited ~40, but I don’t usually log in. I wouldn’t have thought it was important…
posted by Jeremy Dunck on September 5, 2006 #
I find it funny that people are complaining it’s not easy enough to make edits because the big bad super editors revert your writing. Look — wikipedia can’t have it both ways, you need to cite your facts, sometimes that means going to the library to cite a fact you already know to be true. Maybe a google book search, a news search, whatever.
Has anyone noticed the articles about wikipedia in the news lately? They are all scare tactics — wikipedia is a fraud because anyone can make edits! That’s why the editors have to be so tough about what is added, if they really did allow any old thing to be added wikipedia would be completely useless.
So stop complaining, we can’t have it both ways. Either we are strict on each other or wikipedia is garbage. Take your pick.
(And to the person who claimed anon IP’s cant make edits, you dont know what you’re talking about)
posted by tch on September 5, 2006 #
Interesting, but as you comment, the analysis really needs to be expanded beyond a couple articles. Otherwise, you risk generalizing from too small a base.
Speaking of which, I’ll generalize - or at least talk about - myself. The first big thing I did on Wikipedia was editing the Henry James entry up to featured-article status, plus writing a lot of articles on individual works by James. I did much of this work anonymously because I was often too lazy to log in under my account name.
You could say I was your narrow-focus “expert” user at first, the kind of editor you think contributes most of the real content. But then something funny happened. I gradually turned into one of Jimbo’s 0.7% users. Now I make lots of edits to various articles on the encyclopedia, and almost none of them have anything to do with Henry James.
Yesterday, for instance, I worked on an article called Human Waste Project (don’t ask). Yes, I chip in some real content to such articles, but I also do a lot of cleaning up, linking, organizing, citing, etc. As a result, my edit count has started to pile up, and my contributions list has become a hodgepodge of edits on wildly different subjects.
So your two types of editors are not always distinct. Sometimes, a narrow-focus expert can turn into an all-purpose janitor. God help me, it happened to my little old self.
posted by Casey Abell on September 5, 2006 #
Casey: Seth Anthony’s study of admin habits finds exactly what you describe: admins start out doing serious contributions, then become janitors. (He even uses the same term!)
And then, apparently, they burn out and leave (for a while, at least).
posted by Aaron Swartz on September 5, 2006 #
Welcome to the world of the enlightened.
You’ve (re)discovered that most editors don’t edit most pages most of the time, via a different approach from the one Greg Maxwell used.
Thank you very much!
Now if only more people with clue were to actually help me tidy wikipedia guidelines, that’d be nice. :-)
posted by Kim Bruning on September 5, 2006 #
Oh yeah, I’m aware of the pattern. I think Angela Beesley said something similar in an interview somewhere. One caveat: “janitor” may be something of a misnomer. To take another example, I recently “janitored” the article on John Podhoretz. I did a lot of reorganization, straightened out the cites, set up a bibliography, etc.
But I also added a lot of content about Podhoretz’ views on various issues. So it’s not like janitors don’t make content contributions, as well. We shouldn’t get carried away with the distinction between the two types of editors. They really do blend into each other sometimes. For instance, they’ve blended into, well, me.
posted by Casey Abell on September 5, 2006 #
I can’t vote for you — I didn’t make 400 edits, so I don’t count. :-)
<
p>
Dunno if that confirms your suspicion. I can say I contributed substantial text to about 5 articles, 2 of which I just checked, and atleast part of my text is present in the article today.
<
p>
posted by agrajag on September 5, 2006 #
You discovered some fascinating numbers and trends, how you present it a bit as a wall of text: Add some charts in there and you’ll have an internet sensation of an article.
posted by Dennis Forbes on September 5, 2006 #
I don’t think there’s anything you can do to attract occasional contributors to the inner circle of Wikipedia - they just don’t care. They’re happy that they can share some knowledge with the world, and I highly doubt that Wikipedia’s policies lie in their area of interest.
posted by Baczek, an "occasional contributor" on September 5, 2006 #
Your finding pretty much agree with what I’ve found from working on Wikipedia for a year or so. I’m a little surprised that it’s skewed quite as far as you say - I think most newbies start by fixing small problems - then get the confidence in the process to dump their personal knowledge areas into a couple of articles which they proceed to maintain for a while.
Then comes the fork in the road - some get drawn inexorably into the community and make more and more (mostly minor) edits - where others lose interest and go away.
But without doubt, it is impossible for 500 people to have made even the smallest scratch into 1.2 million English language articles. Over the couple of years that Wiki has really taken off, those 500 people would each have had to have written one new article a day in order to have written half of the articles out there.
When you consider the increasing amounts of time those people spend arguing about policy, voting on articles to promote and delete and fixing up style/category/spelling/grammar issues too - that’s just not credible.
Good article - I agree with you.
posted by Steve Baker on September 5, 2006 #
Excellent article!
I agree with most of the things you said.
PS. If I could, I’d vote for you :)
posted by Luke Zapart on September 5, 2006 #
This is an interesting analysis.
This would be an absolutely exellent analysis, if someone would whip up one or two graphs that depict the contrasting metrics, and demonstrate what’s going on.
posted by algal on September 5, 2006 #
Well I’d also vote for you if i could but im not playing whack-an-edit with these 500 subeditors to do that.
I’d question the validity of any editor whom could make 500 changes, what do they know apart from say writing good english.
posted by Bananasin the Falklands on September 5, 2006 #
Unfortunately, it does not work as well as we would hope. Check out the article in The New Yorker…
http://www.newyorker.com/printables/fact/060731fa_fact
posted by dave on September 5, 2006 #
I can tell you from first hand experience that you are correct. What MADE wikipedia great (for the casual contributor) is the ability to “put there two
cents in”.Yes I think the contributors are far more important.The fact is that the editors try to run a dictatorship and delete most of the articles because of format,I thought that was why they are editors in the first place. Now you have to log in to contribute in which about half as people will get involved. So now you have “run of the mill” encyclopedia not an information machine way to go inovation killers
posted by just2speek on September 5, 2006 #
Nope, you don’t have to log in to contribute. I still let a few anon edits slip in occasionally because I forget to log in.
Anyway, I’m pretty exo, which in wikitalk means I concentrate on articles and related content like images, templates and article talk pages. Very interestingly, my edit count profile looks an awful like Aaron’s. (Run Essjay’s edit counter and you’ll see what I mean.)
I like to follow some of the policy debates, and I occasionally get dragged into an article-for-deletion hoo-hah. I also copy the house newspaper, the Signpost, into my user page and make stupid comments on the stories.
So yeah, I’ve gone a little past the fork in the road towards the meta side. But not too far. I still throw in lots of content.
posted by Casey Abell on September 5, 2006 #
Very nice work Aaron!
I was wondering re “random selection” of pages for your tests, … if you randomly select from the entire corpus, versus from the 1000 most popular (on some metric, eg. google rank), versus the 1000 oldest, etc… how the results would vary. Hope you can find some resources to dig further into this.
posted by Dan Brickley on September 5, 2006 #
Speaking of random and non-random article lists, there’s an interesting list at the Signpost site of the 100 most popular articles on Wikipedia. You’ll notice an obvious pattern. Hint: sex sells.
posted by Casey Abell on September 5, 2006 #
It is unfortunate that the qualifications for franchise serve to disenfranchise the most substantive contributors to wikipedia. Since you are attempting to represent them, your campaign is rather handicapped. I wish you the best possible success.
posted by aminorex on September 5, 2006 #
You got my vote, man.
I’d like to see more numbers and charts and more on your methods. Heck, I’d like to rerun your experiment in HuWiki.
posted by nyenyec on September 5, 2006 #
What sort of computing power would be required to get this run over a much larger (10-100%) chunk of the wikipedia?
posted by Jeremy Zawodny on September 6, 2006 #
Wikipedia is past the first phase of its growth, when the emphasis was on seeding content. It has reached sufficient size that parasitic elements start to become noticeable. The emphasis must now be on cultivating an organization of editors that can properly tend to its growth. This will require creating a hierarchy of editors and creating a system so that this hierarchy can be properly created and regulated. For example, an easy complaint mechanism would allow a casual contributor like one of the commenters above to report his problem with an editor wiping out his content to another editor with wider authority, who can adjudicate. Further, he might be given the ability to create a petition making a case for removing an administrator from control of a particular section. This petition might be enforced if he can get enough registered users to vote for it. Those who believe that anything can be done without some sort of administrator hierarchy and easy ways to collect consensus opinions are hoping for a utopian fantasy. Wikipedia may be a good example of the tragedy of the commons, as most contributors, including me, cannot be bothered to look into the administrator selection process. However, Wikipedians must always be aware that someone can fork the content and create their own system with its own editing hierarchy (though the costs of bandwidth and servers will ensure that this is not a spontaneous decision). I suspect that forks to other sites with better administrator processes is what will happen.
posted by Ajay on September 6, 2006 #
No successful forks have happened yet. Fred Bauder, one of the ArbCom members on Wikipedia, tried a fork with Wikinfo, but it never really caught on.
Anybody who tries to impose a hierarchy of editors on a Wikipedia fork will find it hard to lure contributors away from more free-wheeling sites, like Wikipedia itself. That’s what happened to Bauder, and I suspect it will happen to anybody attempting a more rigidly policed fork site.
posted by Casey Abell on September 6, 2006 #
Aaron,
Excellent article, and congratulations on your slashdotting. As per your last article I had gone to wikipedia to vote for you and found out that because I had not made more that 400 edits I couldn’t vote. I was surprised that I didn’t meet the criteria as I’ve been a member for a long time. I decided I must not be a very good member so I doubled (by doing ten) my number of edits instead of voting.
Now reading these points I am all the more motivated to vote for you but have decided that instead of me being a poor wikicitizen wikipedia’s voting system advances an unfair assumption. The same assumption that you unearthed here: that the only people that matter are the large quantity editors.
So I can’t vote for you, but if you do get elected and you do believe what you wrote you should act to change the voting system.
-Andy
posted by Andy Schilling on September 6, 2006 #
You seem to have forgotten considering those users with large numbers of edits and who contribute large amounts to articles. There may be those who are not by your definition “outsiders”. What are the statistics in this instance? This would have made your analysis a little more complete.
posted by on September 6, 2006 #
Intreguing study, Aaron. From my own personal (& incomplete) experience, I have to agree with your conclusion — either I can focus on lots of little edits that concern style, formatting, & so forth, or I can focus on making a few substantial contributions.
Despite having written a number of essays in college, I still find it a challenge to write even 500 words on any given topic for Wikipedia.
So why do some contributors cluster at one extreme or the other in their behavior — & some end up becoming entirely involved in another axis of participation, creating & enforcing policy? I sense material for a Master’s thesis in psychology or sociology here.
posted by llywrch on September 6, 2006 #
A nice article and thx a lot for the insight! Anyhow i do have some concerns: One interesting thing about the wiki-thing in wikipedia is that different tasks (like editing, reviewing, formal qa - to name but a few) are conducted through a single interface and that the complexity of creating an article is therefore driven from the software itself into the flexible organisation of production and publication (so to say, the human factor). Regarding yours and Jimmy Wales research, i think that both approaches show some flaws in terms of this aspect. You both wanted to identify an aspect of quality - Which contributions by whom made up the “essence” (a hard to operationalize term in anyway) of the article by quantitative means. I don’t know if this works at all, but in any case there needs to be some sort of categorization: What kind of edit has been conducted here? Editing, Reviewing, Janitoring (?) etc. Did this edit lead towards the current state of the article or not? The so called “quality” of an article in wikipedia is not defined by absolute means but is the result of the quality demands of the stakeholders, therefore subject to dynamic change. In the end this approach is a tedious and time-consuming work if you want to conduct it for thousands of articles. And also, by the way, not as shiny and reliable as pure quantitative work.
posted by Lars on September 6, 2006 #
“admins start out doing serious contributions, then become janitors.”
<
p>Close. I sort of started on both, got way deep into en:wp administration, then into Foundation stuff, and now I spend my time on content (both copyediting tweaks and writing new short articles) and Foundation.
<
p>I no longer even look at my watchlist. They’re not my articles.
posted by David Gerard on September 6, 2006 #
Very interesting article, and definitely worth reading. I often feel that the ‘Wikipedia community’, such as it is, places too much emphasis on users with a large number of edits (i.e., themselves), and doesn’t consider what those edits actually are. The current voting system is a perfect example of this. I would be voting for you, Aaron, but apparently I have only 85 edits. What I don’t understand is why this makes me less deserving of a say in how Wikipedia is run than someone with more edits.
posted by Alasdair on September 6, 2006 #
I’m also one of the ‘hybrid’ users mentioned above. When I first came aboard 2 1/2 years ago, I went crazy adding content on local institutions and issues, plus topics I know about but didn’t have significant content.
But I only know so much; after a while I was mainly just maintaining what I’d contributed and talking about how else to present it, with other editors who were interested. Once in a while I’ll have a burst of new knowledge - I’ve visited a location or read an article or experienced a unique event - where I can dump a bunch more knowledge. But those are few and far between; 90% of my edits these days are cleanups, formats or comments, of both my work and of existing work that I have interest in and knowledge of.
I think many ‘diehards’ are the same; working to keep things looking good and presentable and encyclopedic to the outside world.
I can’t speak for the ‘knowledgeable newbie’ crowd - I don’t know them well - but we should keep them in mind when making policy decisions. We try to assume good faith and not bite them, as the policy pages say; how realistically they are followed is another matter. That, we will have to leave for the administrators.
I’m still considering my vote. Aaron, you may well be on my final list; I’ll cosider all the issues when it comes to it.
posted by Radagast on September 7, 2006 #
Thanks for doing this analysis, it’s very interesting.
I agree with algal that a graph would be good. I tend to be text-oriented myself, but I know that a lot of people skim the text but look at the figures when reading research.
posted by Bayle Shanks on September 7, 2006 #
Aaron, please provide Jeremy Zawodny with any guidance you can on the work involved. He has the resources to get the work done.
posted by on September 7, 2006 #
Interesting article, I agree with your findings that articles are written mostly by anonymous (expert)contributors.
Mostly they EXPAND the already existing (small) article. I think article creation is less common among anonymous contributors (this would be interesting to find out as well, or maybe there are already statistics on this??).
The value of CATEGORISATION and creating INTERLINKAGES I think should be stressed more in your article, since these improve the encyclopedia and create new insights. categorisation/interlinking effort are probably more often undertaken by the active community (does statistical data exist?).
Another point I’d like to make in my concern with DELETION of whole articles on wikipedia by often the active community: after a small week and some votes of this community on a certain article, the article may be voted to be deleted. This is often decided quantitatively, not qualitatively. Often arguments for keeping the article are not debated; or arguments for deletion are not well founded. Some contributors might be on a wikibreak, come back some time later and find out (or not, even worse) about a deleted article: old deleted pages are (sometimes?) completely inaccessible. This is a loss of content that should be avoided since it frustrates contributors and decreases the quality of the encyclopedia for its users.
Feedback on these concerns is much appreciated! :)
posted by Whirl on September 8, 2006 #
Good article, interesting findings (thanks for the numbers …). I’ll publish a little summary on my blog poiting to your article.
posted by PageTurner.info on September 9, 2006 #
Great article, clear and interesting!
posted by matt on September 9, 2006 #
Contributeur français régulier, je peux confirmer que mon activité se partage entre quelques contributions substancielles dans mes domaines de spécialités et des interventions “d’éditeur” (forme, corrections, vérifications, critiques) beaucoup plus nombreuses. La loi de Pareto s’applique mais dans le sens 80 % des contributeurs pour le contenu et 20 % pour la forme. L’intérêt de Wikipedia n’est pas dans sa ressemblance plus ou moins proche avec les encyclopédies traditionnelles (recueil de l’information, rédaction, validation) qui est inévitable, je ne vois pas comment faire autrement !, mais dans le fait que l’ensemble de ces actions est ouvert à un grand nombre de personnes (potentiellement gigantesque). Car désormais les effets produits sont des régularités, des régulations qui dépassent celles qui étaient jusqu’alors accessibles : le recueil des informations sollicite des dizaines de milliers de personnes et progressivement de plus en plus, ce qui permet de couvrir des domaines qui n’étaient jamais couverts par les encyclopédies traditionnelles, limitées par les ressources de temps et matérielles. Les corrections-validations par des milliers de lecteurs produisent au fil du temps des articles de qualité acceptable qui ouvrent sur un thème et donne les moyens d’élargir la recherche et d’en faire éventuellement la critique (combien de fois suivant un lien externe à WP, j’ai pu critiquer une information et en proposer ou faire la correction dans un article ?). Ce processus est cumulatif dans le temps et accroît encore l’effet “grand nombre” : ne pas en tenir compte serait une grave erreur d’analyse. C’est d’ailleurs une erreur que certains expriment en demandant de manière récurrente des validations par des experts ou des comités restreints : une manière de supprimer l’effet “grand nombre” qui me semble-t-il est la force majeure du processus Wikipedia. Cette force en évolution permanente fait peur à certains, manifestement.
posted by Jean on September 11, 2006 #
This article will have a profound impact on how I approach my own wiki project — http://toastmasters.wikia.com. I’m one of two “super” editors who have made the bulk of edits on the site; I’m now “out of content” — i have, literally, not much more to add. I realise now that I should be approaching so-called Subject Matter Experts and getting them to add content, and letting the rest of us tidy it up for them. Thanks for this insight — you have altered policy on one wiki at least!
posted by Erich Viedge on September 11, 2006 #
Great proactive analysis. Wales’ statement, prior to his “next version”, probably shouldn’t have been so definite..
Last month Jason Calacanis wrote, about the pitfalls of Wikipedia’s syntax and unintuitive interface. Given your findings and as a candidate for the foundation’s Board of Directors do you feel a leveling of the learning curve of the content production process should be of priority? After all, if the content producers are empowered to improve their formatting and structure, the “Top 2%” will have less ground to accumulate edits and thus power.
posted by Angelo Gladding on September 13, 2006 #
Thanks, maybe one of the best articles i ever read about wikipedia. Thank you very much for this result. From my point of view this is confirm to out assumption that there must be a kind of critical mass, after that a wiki system will work. Maybe also from interest our research paper: http://elearningblog.tugraz.at/archives/107
posted by Martin Ebner on September 14, 2006 #
I’ll have to agree with you here. I know that I’ve only made a few contributions to Wikipedia, mostly on subjects that I know enough about to make meaningful contributions. So occasionally I’ll jump on Wikipedia and make a few edits, usually substantial in content (in accordance with the “be bold” imperative of Wikipedia).
Although it hasn’t always happened, largely I find that the pages are monitored by a specific person who end up attempting to erase the edits or substantially change them. Take a look at the Wikipedia page on Jack Thompson, which is basically edited into a hagiography. One of the pages I made a substantial edits to (roughly doubling the length of the article) was then edited back to its initial state by a person with a severe conflict of interest (vice-president of a company who was mentioned in the edits). What it often comes down to is that the people who are compulsively looking out for their own interests, rather than the truth, are willing to dedicate more time on the site squashing what they don’t like. Too bad, really.
posted by Anonymous Contributor on September 14, 2006 #
I wrote an ethnography of a Wikipedia article a bit more than a year ago, looking at the Support Vector Machines article.
Though my conclusions did not directly contradict those above, I did not find that the majority of major edits were made by less active users of Wikipedia. For the Support Vector Machines article, I found that there were two “major editors” - one of whom was a very active Wikipedia user with an account and second was only known by an IP address but knew the norms of Wikipedia very well, indicating he/she too was an active Wikipedia user.
posted by Caroline Moore-Kochlacs on September 15, 2006 #
Wish I’d seen into this essay earlier. This is something I’ve believed for a while, but it’s nice to see someone confirm this by research.
posted by James Yolkowski on September 17, 2006 #
Absolutely fascinating analysis. Well done. Thanks for posting.
posted by Mahdi Gad on September 27, 2006 #
You purchased time on a cluster to analyse “a few articles”! The analysis seems basically correct, however look at people like Charles Matthews and Mav - they make both substantial and substantive contributions.
posted by Rich Farmbrough on October 13, 2006 #
I wish there were more of these analytical studies like your or the famous correspondance on Nature. Wikipedia is becoming too much of an important phenomenon for our society to let people speculate on how it works:we need to know as precisely as possible how it actually works, what kind of impact has on people and how can be improved.
posted by Giorgio on October 24, 2006 #
Respected Sir:
I am a student of class X and wants to complete the project work on numbers 0,1,2,3,4,5,6,7,8 & 9 discovered by whom in ancient times. In fact a writeup is required to be submitted to the Ryan International School, Faridabad.
Kindly help urgently as I am unable to find the site in wikipedia.
An urgent response on above e mail of my father will help me
Thanks & regards,
Vinayak Takru Class X
posted by Vinayak on October 26, 2006 #
i’m not sure, but shouldn’t there be ‘140’ next to the ‘2%’ instead of ‘1400’ in the third paragraph?
posted by jnst on November 16, 2006 #
I think that the regulars play a role that is larger than ‘formatting’. Part of the reason that Wikipedia is not a useless mess is because it is organized. The same collection of information, poorly organized, would be little better than doing a Google search. For many people, in many situations, the current Wikipedia is a more useful way to find certain kinds of information than a Google search is.
Your observation that there are different types of users is spot on, but I think that all types (except vandals) play an indispensible role.
posted by Rafael Garcia on November 16, 2006 #
I’d wager that the statistics are wholly made up and no such code exists. I’m no proponent of Wikipedia, but I call bullshit on the article. From what I’ve read, I believe that Swartz writes fiction.
If what he has written is true, it would be easy for him to prove me and many a skeptic wrong, but I believe he cannot.
JD
posted by J Delphi on December 18, 2006 #
That’s a strong and baseless charge, J Delphi. I’ve provided the code and details on request to those who have asked, other studies have replicated the results, and a larger study is underway.
posted by Aaron Swartz on December 21, 2006 #
I have been a “original content generator” on Wikipedia for some time, but stopped after I’ve become disillusioned with the whole site. I do not see very many users adding original content at all (at least on the articles I’ve worked on), other than just the moving, deleting and rewording things here and there. What we have are the some smuggest, know-it-all editors and admins making snide remarks on the work I have done, but they haven’t done anything to add anything worthwhile. But that’s my experience.
posted by Calvin on December 29, 2006 #
That’s the experience of many people who have tried to participate on Wikipedia, Calvin:
http://nonbovine-ruminations.blogspot.com/2006/10/sigh-wikipedia-needs-better-admins.html http://mail.wikimedia.org/pipermail/wikien-l/2006-October/054949.html http://www.cow.net/transcript.txt http://www.americanchronicle.com/articles/viewArticle.asp?articleID=11109 http://www.roughtype.com/archives/2006/02/community_and_h.php http://www.techybytes.com/is-technology-slowing-down-freedom-of-speech/ http://catallarchy.net/blog/archives/2006/12/20/the-stupidity-of-crowds/ http://www.andycarvin.com/archives/2006/03/wikipedia_blocks_sch.html http://wikipediareview.com/index.php?showtopic=4 http://www.aaronsw.com/weblog/whowritescomments http://pilotguy.wordpress.com/2006/12/20/where-have-all-the-admins-gone/ http://thechrisd.spaces.live.com/Blog/cns!90D096458FBAE74E!242.entry http://stabani.com/archives/2006/247 http://www.techcrunch.com/2007/01/01/wikipedia-bans-qatar/ http://news.bbc.co.uk/2/hi/technology/6224677.stm
posted by Doug Holton on January 5, 2007 #
Most of my contributions have been removed (not modified) and so I have largely stopped contributing.
Often times the contributions are removed and not modified with obscure references to numerous wikipedia policies that seem to make no sense and that are enforced very unevenly across the wikipedia.
At other times, controversial but accurate information is just removed, resulting in a very bland innocuous mostly harmless article. (And not terribly useful.)
The cyberstalking article is a case in point. It is humorous (and sad!) to examine how that article has evolved over time.
posted by Jerry Asher on January 30, 2007 #
Aaron,
I confess. I spend entirely too much time patrolling the web for reading material. But I had to pause to pile on here: this piece in that rare “splendidly good” category. Thanks for the good read.
I’m sorry you didn’t win the election, but, ironically, it’s rather obvious why, eh?
posted by Bill on February 7, 2007 #
What annoys me about Wikipedia is the people’s penchant for rewording other people’s stuff. For instance, I wrote original material on “Impression Fraud” on my site( http://www.sofizar.com/impression-fraud.php ). I was shocked to find that wikipedia entry for impression fraud was a summary of my article with nary a credit. Are the authors of wikipedia just trying to “get done” with everything?
posted by Zafar Khan on February 12, 2007 #
Let’s face it. Few people want to strain themselves of doing the hard and winded research (such as going through books or scouring through newspapers and academic journals, and so on) and writing. It’s easier just to rephrase someone else’s work or use other Web sites as citable “sources” as accessible through Google.
posted by S. on February 12, 2007 #
I wonder if all the people who have had trouble with “mean administrators” really provided valuable content. Most of them never name the articles they worked on so that you can’t check if their claims are true. Just having a point of view doesn’t necessarily mean that it’s reasonable oder widely recognized.
posted by Carbidfischer on February 14, 2007 #
I don’t know. Just what is it with the current zombie-like fascinations with Wikipedia, MySpace, and so on?
Perhaps Wikipedia simply gives people a certain surge of authoritative and administrative power and prestige over others. Those with user names feel entitled to a sense of superiority, compared to a user with a mere IP address and a few original entries.
posted by A.T. on February 20, 2007 #
Great article. One time, I intentionally added factually correct info but with some intentional spelling and grammatical mistakes on a not-so-popular article, but it hasn’t really been touched for nearly 9 months already (and still has not to this day!). All the article received from one editor was a “Citation needed” notice. So where are the lovely by the book editors?
posted by J. Jeffers on February 21, 2007 #
Perhaps they have nothing else to do with their spare time.
posted by ! on February 26, 2007 #
“But on the other hand, the more people view an article the more it is likely to be corrected and balanced for NPOV.”
The more people that view an article the more likely the article is to reflect the views of more people. Sometimes that will tend towards a NPOV and sometimes not. The article will increasingly reflect popular opinion (which may not be neutral at all).
“it isn’t perfect, but in the opinion of many the results are fairly acceptable.”
Perhaps, but not definitive. How do you justify that claim?
When an article can be objective without threatening the views of people it stands a decent chance of being so. Wikipedia is, by its nature, not an objective resource. It is a useful one, though.
posted by Iman Darwiche on March 9, 2007 #
I think the usual problem with Wikipedia is that there is the expectation that someone else will do the work.
posted by Lian Wen-chiu on March 22, 2007 #
I am agree with autors opinion. I think the volunteer authors of Wikipedia articles don’t have to be experts, although some certainly are. They could be anyone! Further, volunteers do not need any formal training before creating a new article or editing an existing article. Many people have created or edited articles in Wikipedia. They live in different countries of earth, from all ages and backgrounds. The autors are called a “Wikipedian” or “Wiki.” The policy to add to the encyclopedia only statements that are verifiable, and not to add original research. To hold the quality high there are special style guide encourages editors to cite sources. I love WIKI
posted by John Miller on April 17, 2007 #
Most of my contributions have been removed (not modified) and so I have largely stopped contributing.
posted by Maca on May 3, 2007 #
Absolutely great analysis and the follow ups on this were also great.
posted by Amit Dixit on May 8, 2007 #
This article is vital; I’ll be sharing it with everyone I know…I only wish I’d found it sooner! It is so reassuring to find that someone else understands — someone with the expertise to illustrate the problems of Wikipedia in a way that cannot be swept aside by the “insiders.” I often find that my own lack of Wiki-protocol knowledge puts me at a disadvantage when arguing points on the talk pages. Thank goodness for WP:IAR (my ace in the hole)! The atmosphere at Wikipedia has become downright draconian; thanks for empowering me to exhale….
posted by Discouraged Editor on May 10, 2007 #
Very good article Aaron. I had this suspicion for a long time myself. Just couldn’t pursue it for various reasons. I am not a particularly big fan of Wikipedia or Jimbo Wales due to their very weird policies of editing. To date I have contributed only one big article and that thankfully survived massive editing. I feel those folks should open up a bit more. It seems way too rigid at the moment. Vandalism is inevitable but the community should grow. Stagnation will not only erode the quality of the site but also lead to its slow and painful death. I hope you read this Jimbo.
posted by Timmy Jose on May 16, 2007 #
Very good article Aaron. I had this suspicion for a long time myself. Just couldn’t pursue it for various reasons. I am not a particularly big fan of Wikipedia or Jimbo Wales due to their very weird policies of editing. To date I have contributed only one big article and that thankfully survived massive editing. I feel those folks should open up a bit more. It seems way too rigid at the moment. Vandalism is inevitable but the community should grow. Stagnation will not only erode the quality of the site but also lead to its slow and painful death. I hope you read this Jimbo.
posted by Timmy Jose on May 16, 2007 #
It’s not the first time in the history of mankind that a Great Thing is taken over by “admins” who don’t understand much of the Initial Idea, but who understand very well how to benefit of it. Hierarchies, structures and rules prevent any further progress and the Great Thing converts into a Serious Thing, that uses the Initial Idea only as a marketing tool.
posted by Severin on May 17, 2007 #
In response to the criticism made by “Carbidfischer”, well, if you have something to benefit from Wikipedia I’m sure you’ll defend them.
Anyhow, I hope the author of the original article could do some similar analyses on the Wikipedia articles on Chinatown, Asian supermarket, and Little Saigon. You see, I wrote a bulk of those articles and over the years I have seen many others just delete here, rephrase there, add a link there, make remarks thither but not much else.
posted by Frustrated one on June 7, 2007 #
It is true that the Editors,or the formatters,are more concerned about the tone,style,category of the wikis.While most part of the world does not find any or even minimum adequate representation,(Even historic monuments and large towns in India do not appear even in the www,what to say of the wikipedia)the concern with formalities does discourage contributions.Contributors do so because they felt something about the topic and the feeling will certainly come out.All of them,or even a few of them cannot be expected to have the attitude and training of encyclopedia editors.The editors appear to be specialists who know only editing and surely they seem to be knowing nothing else.In India,the office typists have the same attitude.They show all the concern for the format only,not for the content,because they do not realise the relative importance of the content.
posted by N.Krishnamurthy on June 9, 2007 #
Dear Aaron,
why do you delete my comment!
You wrote in your article: “Thus his assumption that outsiders contribute mostly vandalism and nonsense. And thus the comments you sometimes hear that making it hard to edit the site might be a good thing.”
If �outsiders� like me give a different viewpoint you think the same way like Jimbo Wales.
Many Regards
Roland
posted by Roland on August 5, 2007 #
Great post Aaron. I’m a CS grad student at Stanford and the sad truth is that the investigation you did as part of your blog post here is far more worthwhile than 99% of the so-called “research” and paper writing that happens in the CS department here.
posted by Jeff Malkes on January 16, 2008 #
Wow, great article. Im glad to hear your findings, after hearing Wales initial findings. It has far reaching implications in society as a whole, not just Wikipedia. How depressing would that have been, that only some .7% of people were capable of any kind of productive contribution to society, with the rest of us just sitting around bickering about the presentation. Though it may seem like that to those with some kind of deep interest and active role in society, really all they are doing is directing everyone elses major contributions. In spite of their self deception to the contrary, their roles as spit shiners is immensely important, and not at all petty. If they could get off of their imaginary high horses to see that, then they probably wouldn’t need the delusion to keep their self image so well bolstered.
posted by Jesse A. Rudolph on August 30, 2008 #
Great article, I find some of the people policing wikipedia near stalinist. And often not knowledgable to an astonishing degree.
posted by ralf on October 14, 2008 #
Thanks for the provocative and interesting analysis. I don’t agree that the editors of traditional encyclopedias concern themselves largely with formatting and presentation. As mediators between the copyeditor and the contributor, their primary objective is to vet content: to assign articles, assure the reader that the contributors have some grounding in an existing literature to establish the authority of their claims, to check facts, and especially to align articles on various subjects with other articles: to remain true to the encyclopedia’s overarching purpose. The insider-editing you describe in your article is largely basic copyediting, but it is far, far narrower than “traditional” encyclopedia editing, and it’s a significant reason why academics and instructors are wary of Wikipedia as an authoritative source as opposed to a window into the minds and ideas of contributors.
posted by Peter Knupfer on January 3, 2009 #
You can also send comments by email.