When the 'wisdom of crowds' turns on itself: IMDB edition

News Type: Event — Seeded on Tue Jul 29, 2008 5:49 AM EDT

Article Source: CNET.com

technology, film, cinema, imdb, the-dark-knight, wisdom-of-crowds, the-godfather, mob-mentality, skewing

The concept of the "wisdom of crowds" is a fundamental building block of a lot of the Web 2.0 services that we see today. While not all of them are built on this core concept, major sites like Digg, Wikipedia, and Mahalo rely heavily on crowds being wise. There have been several instances of this system breaking down in the past, one notable occurrence being the infamous Digg HD-DVD revolt. On occasions like this, a mob mentality tends to take over a very vocal and active segment of a community, greatly skewing the product of their collective "wisdom." We are seeing the latest meltdown of crowd wisdom over at IMDB, the popular movie information database as we speak.

Published to:

Marilyn L's Column, All of Newsvine
Groups: Newsvine Technology, Web 2.0
Regions: none

What's this?

Who's leading the conversation?

This visualization below allows you to see the impact that each user has on the current conversation. The top row contains the group of users who have had the most impact, the 2nd row the group of users who have had the 2nd most impact (et cetera). Users with similar impact are grouped together, and the average score of the group is shown to the left of the group. The author of the article is also shown on the left, in their corresponding group. Each user's score is based on the number of comments the user has made plus the number of votes their comments have received. The scores are calculated relative one another, so while their absolute value is not particularly important, their relative difference does indicate a larger difference in impact on the conversation.

9.6

{"commentId":2306802,"authorDomain":"marilynl"}

Marilyn L

From the article:

I am still a firm believer in the wisdom of crowds. I think that it is a tremendously powerful concept that can be applied to a variety of areas. However, if it is not implemented correctly, you will occasionally get breakdowns like we see here. In the cases of Digg and IMDB, these breakdowns happen infrequently enough that it is not a major concern and does not usually disrupt the use of the site. That said, we have seen, with some consistency, that the system can be broken or manipulated by the actions of a passionate group of individuals, acting as a mob. If these incidents are not largely prevented in the future and become commonplace, we risk a lot of people losing faith in this otherwise very useful system.

{"commentId":2306802,"threadId":"321638","contentId":"1703330","authorDomain":"marilynl"}

2 votes

#1 - Tue Jul 29, 2008 5:51 AM EDT

{"commentId":2308852,"authorDomain":"tcervo"}

gecko85

It's important to realize that IMDB's Top 250 isn't a list of the top 250 best movies of all time, it's a list of the top 250 highest ranked movies by registered users of IMDB. Nothing more, nothing less. When The Dark Knight first made the list (there's a minimum number of votes required) it debuted at #1. It had over 60,000 votes in the first 3 days of release. At that time, The Godfather was unchanged in its ranking. Then the flame wars began. Commenters were slamming fans of the movie, calling them idiots for voting a comic book movie to the top, over a "real" movie like The Godfather. The Dark Knight fans responded, albeit in a childish way.

But you can't discount the initial ranking. Those first votes came en masse during the opening weekend, before the "mob mentality" had a chance to take over, before the backlash had begun.

{"commentId":2308852,"threadId":"321638","contentId":"1703330","authorDomain":"tcervo"}

4 votes

#2 - Tue Jul 29, 2008 11:18 AM EDT

{"commentId":2315482,"authorDomain":"marilynl"}

Marilyn L

I don't doubt your analysis. I'm more interested in the wisdom of crowds' dark side, though. Mob rule isn't usually wisdom, is it?

{"commentId":2315482,"threadId":"321638","contentId":"1703330","authorDomain":"marilynl"}

2 votes

#2.1 - Tue Jul 29, 2008 9:28 PM EDT

{"commentId":2315589,"authorDomain":"tcervo"}

gecko85

Once the mob takes over, then everything goes out the window.

{"commentId":2315589,"threadId":"321638","contentId":"1703330","authorDomain":"tcervo"}

3 votes

#2.2 - Tue Jul 29, 2008 9:40 PM EDT

{"commentId":2315685,"authorDomain":"marilynl"}

Marilyn L

You got it!

{"commentId":2315685,"threadId":"321638","contentId":"1703330","authorDomain":"marilynl"}

2 votes

#2.3 - Tue Jul 29, 2008 9:49 PM EDT

{"commentId":2318364,"authorDomain":"sushicat"}

sushicat

The wisdom of crowds? Please, whether we are talking about crowds on the streets are on the net I think what is happening is a "bully" grown out of proportion.

Even in studies of groups, the crowd, is not be trusted as a benevolent entity. The crowd on the street can and has become our worse nightmare why should the ones on the net be any different?

They want the Dark Knight to be so popular then continue seeing the movie, buy tickets!! But to push the Godfather down on the list is nothing more then a bunch of irresponsible people with a 2-year old mentally.

{"commentId":2318364,"threadId":"321638","contentId":"1703330","authorDomain":"sushicat"}

2 votes

#3 - Wed Jul 30, 2008 6:56 AM EDT

{"commentId":2318719,"authorDomain":"marilynl"}

Marilyn L

I know what you mean, sushicat. However it's a real concept, see what Wikipedia has to say:

The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations, first published in 2004, is a book written by James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group. The book presents numerous case studies and anecdotes to illustrate its argument, and touches on several fields, primarily economics and psychology.

The opening anecdote relates Francis Galton's surprise that the crowd at a county fair accurately guessed the weight of an ox when their individual guesses were averaged (the average was closer to the ox's true butchered weight than the estimates of most crowd members, and also closer than any of the separate estimates made by cattle experts).[1]

The book relates to diverse collections of independently-deciding individuals, rather than crowd psychology as traditionally understood. Its central thesis, that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts, draws many parallels with statistical sampling, but there is little overt discussion of statistics in the book.

Of course, there's also a note that says that the article reads like an advertisement... But in any case, when a crowd is not self-aware, and is big enough, it often makes very good judgments. Unfortunately, on the web, most if not all groups are self-aware.

{"commentId":2318719,"threadId":"321638","contentId":"1703330","authorDomain":"marilynl"}

2 votes

#3.1 - Wed Jul 30, 2008 8:33 AM EDT

{"commentId":2336828,"authorDomain":"sushicat"}

sushicat

Good point Marilyn,

I was just thinking of past incidences where the crowd choose some one (Idol winner (s) and what about that really bizarre singer that all the good people kept leaving but left this guy on stage) and I'm sure when it was left to people to vote on things the votes lacked any reason.

But the averages was interesting. Ok, I'll go read that book. Thanks Marilyn :)

{"commentId":2336828,"threadId":"321638","contentId":"1703330","authorDomain":"sushicat"}

1 vote

#4 - Thu Jul 31, 2008 7:55 PM EDT

{"canLink":false,"threadId":"321638","isPrivate":false}

MARILYN L

Backyard

When the 'wisdom of crowds' turns on itself: IMDB edition

Published to:

FUN STUFF:

COMPANY STUFF:

LEGAL STUFF: