Liveblogging: Infotopia

Just had lunch w/ Cass Sunstein, and he's about to give a talk about his new book Infotopia, about how new ways of sharing and aggregating information. He said that Google is a somewhat daunting place to be presenting on this subject, but he seems like a fairly dauntless fellow.

Partly his book is about the downsides of deliberation as a method of information aggregation. Deliberation has many pitfalls. Other methods include averages (The Wisdom of Crowds suggests that this is good), surveys (which democracy depends on), and prediction markets, which are used at Google.

His interest in deliberation comes from some social science experiments he has done at the intersection of law and psychology, plus technology. Mentions the long tail: the internet allows marketing to niches instead of aggregates, which Chris Anderson thinks is fantastic. However, Sunstein wants to warn about some of the negatives of these things, even though he thinks they are good on net. Some data:

He studied 30 people in Boulder, CO, where they got together to talk about global warming, same-sex marriages, and affirmative action, and what the government should do about them. On the same day, they asked people in Colorado Springs to discuss the same issue. (Note that Boulder is hippy-land, and Colorado Springs is conservative-military land). The idea was to see what happened to the participants point of view. The result was to reinforce the ideological biases. The liberals became more liberal, more worried about global warming, more committed to affirmative action. The conservatives similarly became more conservative. Even in the anonymous statements of views after this deliberation, the diversity of viewpoints decreased dramatically.

Next: American judges (real judges). Appeals courts have panels of 3 judges which make most of the important legal decisions. This consists of a huge natural experiment, since the 3 judges can consist of 0-3 Republicans and the corresponding number of Democrats. So you can look at what happens to judges based on who they are on a panel with. Do like-minded panels become more extreme? It turns out that they do, even though they are discussing technical matters of jurisprudence, not just blowing hot air like the Colorado study. And this holds for pretty much any area of federal law. If an environmental regulation or an ADA case goes before 3 Republicans, it doesn't have a chance, and the same goes for Democrats. The only 2 areas where this effect did not occur was abortion and capital punishment.

The third piece of evidence: juries. They were trying to figure out why punitive damage awards vary so widely. What they found was that there is a wide degree of agreement across race, age, income, etc. about the ordering of badness of misconduct. But there was no agreement about how to put a dollar figure on the harm - people have no basis, so they just make things up. So they did the largest study of mock juries ever conducted, with 500 juries of 6 people each. They found the same thing as in Colorado in terms of "badness" - if people thought something was not too bad before deliberation, they thought it was even less bad after. If they thought it was bad, they thought it was even worse.

But here's the punch line - they did not act this way about awards. The awards after deliberation were at least as large as the lowest award beforehand. In other words, for whatever reason having to do with how our minds work and how we interact, the effect of deliberation on the outcome of the group is profoundly different in these different situations of ideology and assigning numbers. This is a fascinating example, to me, of how we must study these complex systems to understand when they do and don't work - we can't just assume that aggregation improves performance.

Another example is that in a group, information which everyone in the group possesses tends to carry much greater weight than information which one or two people have and must reveal during discussion. Obviously it should carry greater weight, but it carries enormously greater weight - even when the uniquely held information is true and relevant. This effect has been found in multiple domains. It also turns out that lower status people are more reluctant to share unique information, so it seems to have to do with self-silencing based on repuational pressure.

Coming back to the dark side, he says, you can see why having custom-tailored news, reading blogs, and having the niche appeal of the long-tail are not all good. You get an echo-chamber effect which reinforces people's ideological prejudices. He suggests that Google figure out how to deal with this, in their spare time. This is part of the importance of the Public Forum Doctrine of the Constitution, which is that it gives people access to large heterogenous groups, so they can express opinions outside their echo chambers. He thinks it's important to figure out how to do this online.

Surveys: Sunstein asked his Chicago law school colleagues the weight of the winner of the Kentucky Derby, and the number of lines in Antigone, both were very close (Wisdom of Crowds effect). He believes a lot of thinking needs to be done to figure out when crowds are and aren't wise - referenced the Condorcet Jury Theorem, which I blogged about earlier. For example, when he asked his colleagues how many state and federal laws had been invalidated by the Supreme Court, they were completely wrong.

Prediction markets: He thinks they are great. Note that unlike surveys, people will only participate if they think they have relevant information. Furthermore, there is a natural intensity factor where they bet more if they are more confident. Also, they are dynamic - they change continuously over time. As he puts it "Friederich Hayek only had one insight - but it was a great one: that prices serve as information aggregators, and good ones, drawing from many diverse opinions and changing dynamically. And prediction markets have that same power." As a result, and based on the empirical evidence, they are great methods of aggregation.

To summarize: there are lots of methods of aggregation, with various advantages and disadvantages. We need to be careful not to let the internet's ability to cater to the long tail cause us to sort ourselves into echo chambers and reinforce our ideological prejudices. Instead, we should try to harness it's power to aggregate diverse sources of information through means like prediction markets.

Share this