32. Questions 32 through 41 are based on the following passage and supplementary material.
This passage is adapted from John Bohannon, “Why You Shouldn’t Trust Internet Comments.” ©2013 by American Association for the Advancement of Science.
The “wisdom of crowds” has become a mantra of the Internet age. Need to choose a new vacuum cleaner? Check out the reviews on online merchant Amazon. But a new study suggests that such online scores don’t always reveal the best choice. A massive controlled experiment of Web users finds that such ratings are highly susceptible to irrational “herd behavior”—and that the herd can be manipulated.
Sometimes the crowd really is wiser than you. The classic examples are guessing the weight of a bull or the number of gumballs in a jar. Your guess is probably going to be far from the mark, whereas the average of many people’s choices is remarkably close to the true number.
But what happens when the goal is to judge something less tangible, such as the quality or worth of a product? According to one theory, the wisdom of the crowd still holds—measuring the aggregate of people’s opinions produces a stable, reliable value. Skeptics, however, argue that people’s opinions are easily swayed by those of others. So nudging a crowd early on by presenting contrary opinions—for example, exposing them to some very good or very bad attitudes—will steer the crowd in a different direction. To test which hypothesis is true, you would need to manipulate huge numbers of people, exposing them to false information and determining how it affects their opinions.
A team led by Sinan Aral, a network scientist at the Massachusetts Institute of Technology in Cambridge, did exactly that. Aral has been secretly working with a popular website that aggregates news stories. The website allows users to make comments about news stories and vote each other’s comments up or down. The vote tallies are visible as a number next to each comment, and the position of the comments is chronological. (Stories on the site get an average of about ten comments and about three votes per comment.) It’s a followup to his experiment using people’s ratings of movies to measure how much individual people influence each other online (answer: a lot). This time, he wanted to know how much the crowd influences the individual, and whether it can be controlled from outside.
For five months, every comment submitted by a user randomly received an “up” vote (positive); a “down” vote (negative); or as a control, no vote at all. The team then observed how users rated those comments. The users generated more than 100,000 comments that were viewed more than 10 million times and rated more than 300,000 times by other users.
At least when it comes to comments on news sites, the crowd is more herdlike than wise. Comments that received fake positive votes from the researchers were 32% more likely to receive more positive votes compared with a control, the team reports. And those comments were no more likely than the control to be downvoted by the next viewer to see them. By the end of the study, positively manipulated comments got an overall boost of about 25%. However, the same did not hold true for negative manipulation. The ratings of comments that got a fake down vote were usually negated by an up vote by the next user to see them.
“Our experiment does not reveal the psychology behind people’s decisions,” Aral says, “but an intuitive explanation is that people are more skeptical of negative social influence. They’re more willing to go along with positive opinions from other people.”
Duncan Watts, a network scientist at Microsoft Research in New York City, agrees with that conclusion. “[But] one question is whether the positive [herding] bias is specific to this site” or true in general, Watts says. He points out that the category of the news items in the experiment had a strong effect on how much people could be manipulated. “I would have thought that ‘business’ is pretty similar to ‘economics,’ yet they find a much stronger effect (almost 50% stronger) for the former than the latter. What explains this difference? If we’re going to apply these findings in the real world, we’ll need to know the answers.”
Will companies be able to boost their products by manipulating online ratings on a massive scale? “That is easier said than done,” Watts says. If people detect—or learn—that comments on a website are being manipulated, the herd may spook and leave entirely.
Note: The following figure supplements this passage.
Mean score: mean of scores for the comments in each category, with the score for each comment being determined by the number of positive votes from website users minus the number of negative votes
Adapted from Lev Muchnik, Sinan Aral, and Sean J. Taylor, “Social Influence Bias: A Randomized Experiment.” ©2013 by American Association for the Advancement of Science.
Begin skippable figure description.
The figure presents a graph titled “Artificially UpVoted Comments versus Control Comments.” The horizontal axis is labeled “Category of news,” and the following seven categories are listed along the axis, from left to right: “business,” “culture and society,” “politics,” “information technology,” “fun,” “economics,” and “general news.” The vertical axis is labeled “Mean score,” and the numbers 1 through 4 are indicated. The graph shows a white dot with a solid black vertical bar through it and a black dot with a dashed vertical bar through it for each category of news. A key within the graph shows that a white dot with a solid black vertical bar passing through it represents the control comments. A black dot with a dashed vertical bar passing through it represents the artificially upvoted comments. The top of each solid black or dashed bar represents the maximum score for each category of news, and the bottom of each solid black or dashed bar represents the minimum score for each category of news. The white or black dot represents the mean score for either the control or the artificially upvoted comments, with the score for each comment being determined by the number of positive votes from website users minus the number of negative votes. The data represented by each circle and bar, for each category, are as follows. Note that all values are approximate.
Business. Artificially upvoted: maximum, 3.75; mean, 3.2; minimum, 2.55. Control: maximum, 2.25; mean, 2.1; minimum, 2.
Culture and Society. Artificially upvoted: maximum, 3.4; mean, 3.05; minimum, 2.75. Control: maximum, 2.4; mean, 2.3; minimum, 2.25.
Politics. Artificially upvoted: maximum, 2.85; mean, 2.5; minimum, 2.1. Control: maximum, 1.9; mean, 1.8; minimum, 1.75.
Information Technology. Artificially upvoted: maximum, 2.65; mean, 2.2; minimum, 1.8. Control: maximum, 1.8; mean, 1.65; minimum, 1.6.
Fun. Artificially upvoted: maximum, 2.7; mean, 2.35; minimum, 2.1. Control: maximum, 2.1; mean, 2; minimum, 1.95.
Economics. Artificially upvoted: maximum, 2.7; mean, 2.15; minimum, 1.6. Control: maximum, 1.95; mean, 1.85; minimum, 1.75.
General News. Artificially upvoted: maximum, 2.5; mean, 2.15; minimum, 1.75. Control: maximum, 2.1; mean, 2.05; minimum, 1.95.
End skippable figure description.
Question 32.
Over the course of the passage, the main focus shifts from a discussion of an experiment and its results to