Computing the Crowd

crowd_wisdom The best way to determine the weight of an ox without a scale is to “use the crowd.”  So says James Suroweiki in his book, The Wisdom of Crowds.  In this particular scenario he explains that accumulating and averaging the estimates of many individuals (i.e. the crowd), as opposed to taking a single estimate, generally produces an answer closest to the actual weight of an ox in a marketplace.  He concludes with the following explanation:

“…because individual judgment is not accurate enough or consistent enough, cognitive diversity is essential to good decision making.”

Basically, the “wisdom of crowds” is something that emerges when the mistakes of one individual are gradually cancelled out by the accumulated knowledge of the collective.  And as the crowd (along with our ability to track the crowd) expands and is more vocal through the “inter-networking” of our digital devices and the proliferation of social media, it is believed by many that our capacity for wisdom is expanding at a proportionate rate.

In the digital age, the crowd generates “wisdom” consciously in Wikipedia pages, or unconsciously by contributing to Big Data.  It is interesting in both cases to understand how the crowd is defined and how wisdom is understood.

Wikipedia:  An Encyclopedia of, by, and for the People.

wikipedia In 2001, Jimmy Wales and Larry Sanger launched Wikipedia because they wanted an encyclopedia of the people, by the people, and for the people.  That is, they wanted everyone, not just a few professionals behind closed doors, to have a hand in creating categories and contributing content for an online, open-source, and free encyclopedia.  At the heart of their vision was the conviction that an overwhelming quantity of user opinions would gradually produce a quality product.  Steel sharpening steel, right?

Although the quality of Wikipedia’s product is under continual scrutiny, its hold on our culture is clear.  Struggling to pull up the name of that actor on TV?  Not sure what happened in 1776?  Need a brief synopsis of To Kill a Mockingbird before that book report is due?  A quick Wikipedia search and the answers to these and many more questions are waiting for you in a neatly structured web page.

Today there are 25 million free articles available on Wikipedia in over 250 languages.  Over 35 million anonymous contributors provide its content and it is understood to be the sixth most visited website, tracking upwards of 80 million monthly hits in the United States.  Yet we love to hate Wikipedia because ordinary people, and not professionals compile it.  Despite this common criticisms against the site, studies have indicated that this open-source encyclopedia contains less factual errors than its “closed” competitors, such as Encyclopedia Britannica.

Interestingly, a primary reason for this finding is that an erroneous entry on Wikipedia is spotted and corrected instantly by professionals who are determined to maintain factual accuracy on topics close to their heart.  If someone finds an error in Britannica, on the other hand, it will not be dealt with until the next edition is printed.

The Wisdom in Wikipedia’s Crowd

 The-Thinker-WikipediaIndividuals that celebrate crowd wisdom often point to Wikipedia as a great example of collective knowledge that gradually erases the mistakes of individuals over time.   But I do not believe this is a correct assessment of the online encyclopedia because difference of opinion is preserved, not discarded. It is possible to view the changes made to any entry.  In this respect, the crowd of Wikipedia contributors is not some singular entity, but really is a collective of individuals who are passionate and more often than not, proficient in a particular area.  A wise way to use Wikipedia is to uncover these different opinions and attempt to understand how and why people diverge in their interpretation of particular events.

Wikipedia is rarely used this way and the “wisdom” it promotes is little more than a thoughtless fusion of knowledge and information.  Of course, this is not a problem created by Wikipedia.  TS Eliot already lamented the fact that the pre-Internet sea of information only submerged our capacity for wisdom, lamenting in his Choruses from the Rock:

Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?

Yet, today we happily bathe in this sea, believing that wisdom can be attained when we have the right tools for navigating our way through information even faster.  The result is that the crowd is no longer valued as a collective of individuals but rather as a singular body of data, which brings us to the second example of crowd wisdom: Big Data.

Big Data and the Flu

google

Nipping pandemics in the bud before they get out of hand is one goal of the US Center for Disease Control and Prevention (CDC).  This is tricky because tracking the spread of pandemics takes far more time than it does for a pandemic to spread.  Consider the flu.  A doctor may not be consulted until an infected person has been sick for multiple days.  The doctor’s diagnosis may not be accessible information for days after that.  At this point, the entire continent may be infected by a particular strain of the flu.

The CDC needs need data.  And it needs it fast.  In their new book, Big Data:  A Revolution that will Transform how we Live, Work, and Think, authors Viktor Mayer-Schonberger and Kenneth Cukier show that Google’s search engine is a better option for flu prediction (and prevention) than the CDC.  How?  It turns out that most people will Google symptoms of the flu before visiting their doctor.  Since Google processes upwards of 3 billion search entries every day, they can access data on the flu quickly by extracting 50 million of the most common flu search terms.

The end result is a detailed map of the flu’s emergence in specific locations around the world.  A search engine that billions of people use combined with the tools to assess search data quickly allowed the CDC to undercut the common limits of time and space when determining the spread of disease.

big-data_50291c3b16257

The Wisdom of Big Data

Without realizing it, the crowd produced valuable information just by “Googling.” This generated an important data set, known as Big Data.   Recently, NPR’s Cosmos and Culture blog featured a short write up by Alan Frank on Big Data.  Here is how Frank describes both the emergence of and our growing faith in Big Data:

At the intersection of the Internet, digital recording technologies and a profusion of small-scale wireless sensors comes an exponential increase in stored data. We’re recording information of every kind: tweets, engine temperatures, Facebook photos, stock trades, grocery store purchases. These data points pile up, billions upon billions, trillions upon trillions, recording our lives and the life of the world we inhabit. They are very, very valuable to people with the ability to see patterns in, and extract insight from, the blizzard of information blowing out of the cloud.  Ultimately, the promise of Big Data is the ability to understand (and control) a seemingly chaotic world on levels never before imagined. (emphasis mine)

Now, the flu scenario shows us that Google’s ability to compute the crowd and process big data is valuable.  It may even be a reason for hope. But again there is something problematic with this conception of the crowd and wisdom.

The crowd according to Google is no longer a collective of individuals but only bits of data.  The crowd is valuable as fragments of data, not as human beings with creative potential as is the case with Wikipedia. Jaron Lanier summarizes this view succinctly:

A fashionable idea in technical circles is that quantity not only turns into quality at some extreme scale, but also does so according to principles we already understand.  Some of my colleagues think a million, or perhaps a billion, fragmentary insults will eventually yield wisdom that surpasses that of any well thought out essay, so long as sophisticated secret statistical algorithms recombine the fragments.

This comment also taps into the idea of wisdom promoted by Big Data fanatics.  Wisdom is nothing more than data processing at previously unimaginable levels.  These individuals might respond to TS Eliot that the wisdom we lost in knowledge and information will be retrieved with the perfect algorithm.

Conclusion:  Biblical Crowds and their Wisdom

Christians attempting to navigate their way through our digital age might question the importance of an article devoted to the “wisdom of crowds.”   But consider the cultural pull of Google and Wikipedia as they are now first responders to many of our (and our children’s) questions about life.  Their authority as the guardians of information resembles spiritual and scientific institutions.  And the allure is found in the “open-ness” of both services.  That is, they invite us to connect with, learn from, and teach one another.  Our faith, hope, and love are to be placed (consciously and unconsciously) in the crowd.  This is the wisdom of our age.

In the gospel of John we encounter the crowd a few times.  For example, in John 12: 12 – 13, it is the crowd that celebrates the arrival of Jesus.  We read:

 “The next day the large crowd that had come to the feast heard that Jesus was coming to Jerusalem.  So they took branches of palm trees and went out to meet him, crying out, ‘Hosanna!  Blessed is he who comes in the name of the Lord, even the King of Israel!’”

 Although the crowd is calling Jesus “King,” the collective mind changes quite rapidly.  By John 18: 37-40 we read:

 “Then Pilate said to him, “So you are a king?”  Jesus answered, “You say that I am a king.  For this purpose I was born and for this purpose I have come into the world – to bear witness to the truth.  Everyone who is of the truth listens to my voice.”  Pilate said to him, “What is truth?”   After he had said this he went back outside to the Jews and told them, “I find no guilt in him.  But you have a custom that I should release one man for you at the Passover.  So do you want me to release to you the King of the Jews?”  They cried out again, “Not this man, but Barabbas!”  Now, Barabbas was a robber.”

jesus-of-nazareth-3Pilate, caught between the crowd and Jesus Christ, is confronted by one question:  What is truth?  Not realizing that he is standing before the author and embodiment of truth, Pilate decides to turn his back on the individual, wash his hands of the matter, and pragmatically extract truth from the crowd.  When I try to consider the limits of “collective wisdom” I am reminded, and humbled, that it was the wisdom of the crowd that nailed the truth to a tree.

Advertisements

3 thoughts on “Computing the Crowd

  1. This is a very well-considered post, and I think you make a very good point in it: I think more often than not, the crowd, for sheer volume, drowns out the one, and in the one often lies the crucial piece of wisdom or understanding that can move us, as a race, forward. And Wikipedia definitely proves that over-abundance of info does NOT equal wisdom, truth, or even knowledge.

    I will say one thing, though: as it is dangerous for the Christian seeking truth and wisdom to look for it in the “crowd” (as I read you, “crowd” equals “world”–perhaps I read you wrong?), it is equally dangerous for the Christian seeking truth to defer to the “crowd” in the Church. A multiplicity of Christian voices also often does not equal either truth, wisdom, or knowledge…

    • The crowd is any collective of individuals, both inside and outside of the Church. And I would agree that history proves both groups are prone to arrive at conclusions that are neither true or wise, time and again. After all, it was the Church that bore the brunt of some of Jesus’ harshest criticisms, and the Church was certainly present in the crowd that wanted Him dead.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s