Competition of Experimentation?
It should be clear to anyone who has studied the topic that the way to drive innovation forward is to have lots of small groups of people each trying different things to succeed. In Guns, Germs, and Steel, for example, we see that certain societies succeed because geography breaks them up into chunks and prevents any one person with bad ideas from getting control of too much, while other societies fail because their whole territory can too easily be captured by an idiot.
It might at first seem more efficient to let the whole territory be captured by a genius, but a moment’s reflection will show that there are few geniuses whose brainpower can match the combined results of many independent experiments. This has fairly obvious applications to business and other fields, but for a moment let’s just think about the concept itself.
This idea is often presented as a defense of competition and the capitalist market system that embraces it. Innovation only happens, such people say, when lots of people are competing against each other for the prizes of success. In a communist country, where Big State decides what will be worked on and how, there is no incentive to innovate. Only in a country like ours, where the victor gets the spoils, can new technology be developed.
And yet we also know that competition is a terrible way to get people do well. In No Contest: The Case Against Competition (now out in an elegant 20th anniversary edition) we see dozens of studies that show that, by all sorts of metrics, people’s performance (and enjoyment) goes down when they are forced to compete. Even worse, it goes down most notably for creative tasks — precisely the kind of thing involved in innovation.
How do we resolve the contradiction? The key is to notice that competition, especially market competition, isn’t the only way to encourage experimentation. And that’s often hard to do, because typically market competition is treated as the only sensible form of competition and competition as the only sensible form of experimentation. But that’s not at all the case.
Instead of providing a prize for winner, we could provide rewards to everyone who tries. And that actually makes sense — not only because prizes also decrease productivity and creativity — but also because, when it comes to experimentation, it’s not really your fault if the experiment doesn’t work. In fact, we want to encourage people to try crazy things that might not work, which is exactly why rewards are so counterproductive.
But even if you don’t give an explicit prize, competition is still unhealthy. Contrary to what the apologists for market theology would like you to believe, people do not work better when they’re terrified of the guy next to them finding the solution first. Which is why we should look at this as simply experimentation, not competition.
Experimentation can certainly be carried out cooperatively. Imagine many different scientists in a lab, each trying different ideas during the day, swapping notes and tips over lunch, perhaps joining together to form small groups for certain experiments, or perhaps helping with little pieces of other projects in which they have particular expertise. Each scientist may disagree on which is the right direction to pursue, but that doesn’t make them enemies.
That’s the way that science progresses. And, if you let it, other things too.
You should follow me on twitter here.
December 7, 2006
Are you sure you aren’t over-generalizing? Although rewards and prizes may decrease productivity when it comes to learning things, or working on intellectually sexy projects, that can’t be true for everything. And aren’t most things (even on the innovative side) not all that intellectually interesting?
Or am I misunderstanding what you’re implying we should apply these ideas to?
posted by Andrey Fedorov
on December 7, 2006 #
The studies I cite are pretty general. Certainly there must be exceptions, but if you read the books you’ll see they’re much rarer than you’d think.
posted by Aaron Swartz
on December 7, 2006 #
As Guns, Germs and Steel also points out in the chapter on domesticable animals, there is no factor which makes an animal domesticable. But there are any number of different factors which alone will make the animal un-domesticable.
Similarly, there is no one factor that will encourage innovative behavior. A certain percentage of the population is creative and productive, and doesn’t really need any incentive to be so. However, there are many factors which will suppress this natural urge.
A big one is the apathy of one’s peers. Another big one is the lack of resources (money, tools or connections) which a person needs to take something from idea to experiment to result. The need to fulfil the daily grind to earn one’s bread might seem like another one, but it depends on how much of one’s enery is lost to this activity. Einstein worked in the patent office, for example.
posted by Gordon McNutt
on December 7, 2006 #
“Each scientist may disagree on which is the right direction to pursue, but that doesn’t make them enemies.”
That’s sort of a fantasy-land idea of science. Rivalries among scientists and mathmeticians are just as bitter as any other. (Newton/Liebniz, Teller/Oppenheimer, Kronecker/Cantor, and so on).
Aside from that, I have two questions about your thesis:
- What mechanism do you propose for “rewarding everybody who tries”? How hard to you have to try before qualifying for the reward?
- If market competition causes people to be risk-averse, as you claim, then why do so many people persist in taking risks like the one you did by joining reddit?
posted by Mark
on December 7, 2006 #
I’m a bit driven to distraction by the discussion of what capitalists are supposed to believe. Even the most ardent free-marketeer would have to admit that the Soviet Union, the epitome of top-down decisionmaking, was still able to develop many new technologies, particularly in the areas of nuclear weapons and space travel. The question is which approach is optimal. It also represents a poor understanding of market competition to insist that being “forced to compete” and the terror that entails is necessarily preferable to being simply “free to compete,” which can be a laid-back affair. And many rewards are non-monetary. You may help develop a lasting idea for little or no pay, yet still be rewarded in prestige. Rewards also include the much greater satisfaction you derive from helping to develop something that works than something that doesn’t.
posted by Mike Sierra
on December 7, 2006 #
we could provide rewards to everyone who tries
Trying something new is its own reward.
The only way you can stop people from trying to invent new sources of value is to punish/kill them for trying.
And there are plenty of social organizations that will punish/kill you for trying.
If you don’t sell yourself into wage-slavery, and if you are not currently in the cross-hairs of the thugs with machine guns, you will notice resources to nurture and disseminate discovery, far beyond what Galileo had access to.
Otherwise, you have to keep your head down, and hope you are found out only after it is too late.
But I agree with you, the best that can be said about the competitive capital markets (considered along with the societal, legal, and governmental frameworks that actually make the market possible) is that they punish only a percentage of people who are trying to invent new sources of value. They don’t manage to punish 100%. Yet.
posted by manuelg
on December 8, 2006 #
I disagree.
Consider the subject of automated vehicles. This is somewhat of a “sexy” field, and many groups have worked on it over time, with mixed results.
Enter the DARPA Grand Challenge. Many teams, lots of work, 2 years, and now we have vehicles that can travel 100 miles autonomously. There are limitations, but this contest had a huge impact on the progress in this field.
The Ansari X-Prize worked equally well, and several of the contestants are still developing their projects, even though the contest was won some time ago.
I haven’t read the books cited, but these are two rather significant counter examples.
posted by mp
on December 8, 2006 #
Aaron, I find that you generally have a utopian vision of how science works. I’ve been to labs that so hyper-competitive that the lab boss assigns the same project to different members of the lab, where it is expected that whoever gets the project working first gets all the glory.
In science, publishing first is so important, because sometimes you are unable to publish if the result is too close to something that is already published. I’ve seen grad students lose years of work by being pipped to the finishing line in publishing.
I can name you rivalries in my field, where bombastic assholes hold important positions in university hiring committees, grant committees. These people hold real power in terms of providing resources and recognition to advance their competitors career. This holds for advancing the interests of vulnerable postdocs as they make their up the acadmeic ladder.
And science is becoming way less collegial and more cut-throat. Across the board, it’s expected that only 10% of postdocs will get a professorship. And that’s a conservatively high figure.
posted by Bosco
on December 8, 2006 #
What Bosco said might be ‘tradition’ of university.
Academic Charisma and the Origins of the Research University by William Clark
In New Yorker there was a review article - last month? by Anthony Grafton of this book and , also of course it’s on Amazon,
Noel Malcolm wrote same thing about the filed of Liberal Arts.
posted by follwo up
on December 8, 2006 #
sorry, mistyped.
filed > field
posted by follow
on December 8, 2006 #
You can also send comments by email.
Comments
Are you sure you aren’t over-generalizing? Although rewards and prizes may decrease productivity when it comes to learning things, or working on intellectually sexy projects, that can’t be true for everything. And aren’t most things (even on the innovative side) not all that intellectually interesting?
Or am I misunderstanding what you’re implying we should apply these ideas to?
posted by Andrey Fedorov on December 7, 2006 #
The studies I cite are pretty general. Certainly there must be exceptions, but if you read the books you’ll see they’re much rarer than you’d think.
posted by Aaron Swartz on December 7, 2006 #
As Guns, Germs and Steel also points out in the chapter on domesticable animals, there is no factor which makes an animal domesticable. But there are any number of different factors which alone will make the animal un-domesticable.
Similarly, there is no one factor that will encourage innovative behavior. A certain percentage of the population is creative and productive, and doesn’t really need any incentive to be so. However, there are many factors which will suppress this natural urge.
A big one is the apathy of one’s peers. Another big one is the lack of resources (money, tools or connections) which a person needs to take something from idea to experiment to result. The need to fulfil the daily grind to earn one’s bread might seem like another one, but it depends on how much of one’s enery is lost to this activity. Einstein worked in the patent office, for example.
posted by Gordon McNutt on December 7, 2006 #
“Each scientist may disagree on which is the right direction to pursue, but that doesn’t make them enemies.”
That’s sort of a fantasy-land idea of science. Rivalries among scientists and mathmeticians are just as bitter as any other. (Newton/Liebniz, Teller/Oppenheimer, Kronecker/Cantor, and so on).
Aside from that, I have two questions about your thesis:
posted by Mark on December 7, 2006 #
I’m a bit driven to distraction by the discussion of what capitalists are supposed to believe. Even the most ardent free-marketeer would have to admit that the Soviet Union, the epitome of top-down decisionmaking, was still able to develop many new technologies, particularly in the areas of nuclear weapons and space travel. The question is which approach is optimal. It also represents a poor understanding of market competition to insist that being “forced to compete” and the terror that entails is necessarily preferable to being simply “free to compete,” which can be a laid-back affair. And many rewards are non-monetary. You may help develop a lasting idea for little or no pay, yet still be rewarded in prestige. Rewards also include the much greater satisfaction you derive from helping to develop something that works than something that doesn’t.
posted by Mike Sierra on December 7, 2006 #
Trying something new is its own reward.
The only way you can stop people from trying to invent new sources of value is to punish/kill them for trying.
And there are plenty of social organizations that will punish/kill you for trying.
If you don’t sell yourself into wage-slavery, and if you are not currently in the cross-hairs of the thugs with machine guns, you will notice resources to nurture and disseminate discovery, far beyond what Galileo had access to.
Otherwise, you have to keep your head down, and hope you are found out only after it is too late.
But I agree with you, the best that can be said about the competitive capital markets (considered along with the societal, legal, and governmental frameworks that actually make the market possible) is that they punish only a percentage of people who are trying to invent new sources of value. They don’t manage to punish 100%. Yet.
posted by manuelg on December 8, 2006 #
I disagree.
Consider the subject of automated vehicles. This is somewhat of a “sexy” field, and many groups have worked on it over time, with mixed results.
Enter the DARPA Grand Challenge. Many teams, lots of work, 2 years, and now we have vehicles that can travel 100 miles autonomously. There are limitations, but this contest had a huge impact on the progress in this field.
The Ansari X-Prize worked equally well, and several of the contestants are still developing their projects, even though the contest was won some time ago.
I haven’t read the books cited, but these are two rather significant counter examples.
posted by mp on December 8, 2006 #
Aaron, I find that you generally have a utopian vision of how science works. I’ve been to labs that so hyper-competitive that the lab boss assigns the same project to different members of the lab, where it is expected that whoever gets the project working first gets all the glory.
In science, publishing first is so important, because sometimes you are unable to publish if the result is too close to something that is already published. I’ve seen grad students lose years of work by being pipped to the finishing line in publishing.
I can name you rivalries in my field, where bombastic assholes hold important positions in university hiring committees, grant committees. These people hold real power in terms of providing resources and recognition to advance their competitors career. This holds for advancing the interests of vulnerable postdocs as they make their up the acadmeic ladder.
And science is becoming way less collegial and more cut-throat. Across the board, it’s expected that only 10% of postdocs will get a professorship. And that’s a conservatively high figure.
posted by Bosco on December 8, 2006 #
What Bosco said might be ‘tradition’ of university.
Academic Charisma and the Origins of the Research University by William Clark
In New Yorker there was a review article - last month? by Anthony Grafton of this book and , also of course it’s on Amazon,
Noel Malcolm wrote same thing about the filed of Liberal Arts.
posted by follwo up on December 8, 2006 #
sorry, mistyped.
filed > field
posted by follow on December 8, 2006 #
You can also send comments by email.