One of my early posts was about idea killers and idea helpers. Idea killers are common phrases that kill creativity at its origin. Recently, Matt May wrote a piece called Mind of the Innovator: Taming the Traps of Traditional Thinking, where he identified ‘Seven Sins of Solutions,’ routine patterns of thinking that prevent people from being creative. He suggests that idea stiffling is the worst sin being the most destructive. He illustrated the point by a nice experiment:
At the off-site, there were about 75 people of varying degrees of seniority, ranging from field supervisors to senior execs. i gave the assignment, one of those group priority exercises whereby you rank a list of items individually and then as a group and compare (sort of a “wisdom of crowds” exercise to show that “we” is smarter than “me”). this specific exercise required you to rank 25 items with which you’ve crashed on the moon in relation to how important they were to your survival. nasa had compiled the correct ranking, so there was a clear answer.
I did the exercise with a twist. at each table i put a ringer. i gave the lowest-ranking person the answer. it was their job to convince the command-control types they knew the right answer.
During the group exercise, NOT A SINGLE CORRECT ANSWER GOT HEARD.
After debriefing the exercise in the regular way, i had each person to whom i had given the correct answer stand up. i announced that these individuals had offered the right answer, but their ideas had been stifled, mostly due to their source and stature and seniority, or lack thereof.
I wish i had a camera to catch the red-faced managers.
This is a good example illustrating repeated failure of collective intelligence. Matt suggests that in collective problem-solving workshops groups discuss the right answer and still commonly propose the wrong one as the chosen solution, “because members second-guess, stifle, dismiss and even distrust their own genius.”
Collective problem solving involves iterated innovation and selection of solutions. In his experiment, Matt decoupled the two by ensuring that the right solution was injected into the pool of solutions that group considered and yet he repeatedly observed that the group rejected the right solution. Apparently, the group evaluation of ideas was seriously biased towards accepting the inferior ideas of the senior members at the expense of other ideas, i.e. the senior status of the idea source overweighted the intrinsic merits of the idea. This is an example of subjective selectionist bias. Another common type of bias is temporal bias. For example, solutions proposed earlier can be preferred to solutions proposed later (or the other way around).
We can see these sources of bias working in many “collective intelligence” web 2.0 platforms, where people are supposed to select the fittest among several versions of the content based on the merit of the content. However, in reality, the selection is heavily biased by other factors that has little to do with the quality of the content. Yahoo Answers, for example, presents the answers for voting in the order they were received. The earliest solutions get at the top of the list and receiving a disproportionally high number of votes regardless of their merit. Wikipedia exhibits the opposite kind of temporal bias where the last edit always wins, at least until it is reverted by someone. The revert decision itself is heavily biased by the status of the person who made the edit (e.g. anonymous/registered/admin). Majority of web 2.0 sites make the information about the content author readily available. This results is selection bias towards the content contributed by senior members of the community in the same way as in Matt’s experiment. This is the mechanism ensuring that any Kevin Rose’s submission ends up at the front page of Digg, while the contribution of an ordinary Digg user is unlikely to get there regardless of its merit.
When I was designing 3form website 10 years ago, my primary goal was to select the fittest solutions regardless of who submitted them, i.e. reduce the decision error and subjective/temporal biases that contribute to this error. As a result, 3form interface isn’t showing the names of the authors of submissions to be evaluated (like in a blind peer review). The presentation order of solutions is randomized to reduce temporal bias, so every solution has an equal chance to be placed at the top of the list for evaluation.
The Matt May manifesto is found thanks to Guy Kawasaki post The Seven Sins of Solutions.
my favorite blog post of yours yet — excellent.
[...] This returns us back to the topic that I discussed in my previous post Bugs of collective intelligence: why the best ideas aren’t selected?. The common failures of collective intelligence clearly suggest that it is not a phenomenon that automatically emerges once someone set up a shared space like wiki and brought it to the attention of many people. It requires understanding of the dynamic of this systems to make them work, and this is especially true with wikis. There is still serious research to be done on the factors that make different collective intelligence methods effective. It is beyond the scope of this post, but here I want to give some hints into why some wiki-based projects perform poorly. [...]
Interesting concepts & info - Thanks!
Completely FWIW… Though probably contrary to purest aims of Collective Intelligence, I wonder if the inclusion of social engineering &/or behavioral psychology into the design & implementations might not improve many (most?) aspects of performance? In what seems at the moment a collection of ironies…
I would think that the main goal of Collective Intelligence as a whole would be to overcome the burdens and inefficiencies of societal structures as relates to creative, best solutions. Yet the very behaviors/structures Collective Intelligence would overcome persist, perhaps insist on imposing the same limitations on Collective Intelligence efforts themselves. A partial solution might be found based on frankly tricking (the polite word would be encouraging) those involved to participate as desired, in a way more free of constraints, while maybe adding some sort of anti-biasing for tweaking.
Implementing such a solution would of course likely be analogous to jabbing the back end of an angry bull with a hot poker should it become known. And you’d be introducing something almost like a self-destruct mechanism, providing the means for an over-riding bias to be purposely introduced — a lot of energy would have to be spent designing safeguards to preserve overall integrity.
[...] Korean Naver played a key role in popularizing the concept of knowledge market. Naver, however, was not the first knowledge market in Korea. DBDiC offered analogous service as early as Oct 2000. DBDiC presumably developed its technique independently from 3form, but shares similar architecture, including general structure and credit assignment system. There are two key differences of DBDiC technique from the one of 3form: (1) the identity of the author of the solution biases evaluation of the solution, i.e. high status of the author can lead to accepting the inferior answer as the best despite the presence of the better answer contributed by a person with lower status, (2) answers that are positioned earlier in the list are more likely to be read, chosen, and evaluated, i.e. a great answer later in the list can easily be overlooked. These differences resulted in subjective and position specific biases in solution evaluation system (see also my previous post .Bugs of collective intelligence: why the best ideas aren’t selected?). I could speculate that if DBDiC designers were more familiar with 3form service, they could have avoided those undesirable biases that later propagated into every knowledge market platform created subsequently. [...]