Decentralized Systems Scale Better

Centralized systems can be quite alluring. They're easy to analyze because its clear how they work, and when small, they can be very efficient. Unfortunately, they don't scale well. As the system grows, there are more and more decisions to be made, yet still the same small group of people making them. The standard answer is a multi-layered bureaucracy, but decision quality degrades because of the distance between the decision makers at the top and information holders at the bottom.

In decentralized systems, control and information are spread throughout a network of individual agents, and its not always clear exactly how they work. However, the supply of decision-making ability does scale up naturally with demand. More agents means more decisions to be made, but also more people to make the decisions.

The Strangerhood, episode 1, opening shotFile-sharing systems offer a similar, real-world example, where the relevant resource is bandwidth rather than decision-making. Case in point: Rooster Teeth Productions , makers of Red vs. Blue (a TV-like series created using a video game rendering engine) got slashdotted today for releasing the first episode in a new series, The Strangerhood, based on The Sims 2. Their website is very slow because of the slashdot effect, which would make it hard to get the new episode, except that someone in the comments thread posted a BitTorrent seed.

p2p network diagramBitTorrent is a P2P system based on individual files rather than an entire network. You connect to everyone else who is downloading the same file, and you trade the pieces you have for the pieces you don't. So bandwidth supply automatically scales with bandwidth demand. More people downloading means more people uploading. The seed file is very small, because it just contains some meta-information, so hosting it at a centralized location is no big deal. Using BT, I was able to get the 33MB new episode in minutes.

This offers a clear demonstration of the scaling effects. The centralized system (a webserver) performed poorly when an unexpectedly high load occured, yet the decentralized P2P system worked perfectly. Because we evolved in an environment of small groups, and people like to be in charge, we have a natural tendency towards centralized solutions. But in a dynamic world with systems containing millions of times as many people as any prehistoric tribe, these approaches perform poorly.

I'll point out the analogy to blogs vs. MSM, and leave it to the reader to extend the idea to current social, political, and economic systems.

Update: Hit-n-Run uncovers a similar example about watching the Jon Stewart/Crossfire confrontation, which I was able to download in less than 10 minutes using this BT link. Several times as many people have downloaded the episode as watched it live.

Share this