Methodology matters when determining intelligence

A recently launched website, imaginatively titled “Booksthatmakeyoudumb,” presents the results of a curious query: Is there a correlation between people’s intelligence and the sort of books people read? The site’s creator, Virgil Griffith, is a graduate student at Caltech and has a history of interesting data mining projects: he was also responsible for the WikiScanner, a site that correlated IPs of users making Wikipedia edits to the domains they belonged to and set off a number of scandals.

The site’s methodology is simple: Take the top books listed by students in 1352 college networks on Facebook, and limit it to the most popular 100. Then, find the colleges’ average SAT scores based on College Board data and chart the books thus associated with high SAT scores.

The results are, to say the least, interesting. The top three books on the list are Lolita, 100 Years of Solitude, and Crime and Punishment, while the bottom three are Addicted, The Color Purple and Zane. Other notable matches include the Holy Bible at number 92 out of 100, and the bestselling “book” titled I Don’t Read was in 87th place.

The books popular at the schools with the lowest scores fall into the erotica, African-American literature, dystopian and religious genres. Meanwhile, at the top are philosophy, sci-fi/fantasy and classics.

I’m sure by this point many of you are thinking “This is ridiculous; I’m way smarter than those people with a 909 SAT score who put the Bible in their favorite books, and it’s one of mine.” But the list is not generated with particularly extensive scientific rigor, and the methodology has some obvious holes.

Most critically, the average SAT score of a university is not a very good reflection of an institution’s actual quality or the intelligence of their students. And the books listed are only based on the much smaller subset of people at a university who both use Facebook and enter a list of their favorite books, all spelled correctly. All together, there is a reason the information is on a website and not in a scientific journal.

The results are entertaining mainly from an idle curiosity perspective and should not be taken as insulting to particular groups. But it seems inevitable that some will take the site’s contents quite negatively. Any time information can be used to draw clear implications about the IQ of specific groups of people (as these results inevitably do), a major pain point seems to be triggered.

A similar debate took place after James Watson, the Nobel laureate who, along with Francis Crick, discovered DNA, said in an interview with the Times of London that “all our social policies are based on the fact that [Africans’] intelligence is the same as ours—whereas all the testing says not really.”

Watson’s comments set off a firestorm of criticism, and understandably so—as a prominent biologist his comments are given great credibility. They were picked up by a number of authors, and some opted to, at least for the sake of argument, consider the evidence and its implications.

Others disagreed vehemently, but the most interesting rebuttal came from Malcolm Gladwell, the popular author of The Tipping Point, in The New Yorker magazine. He pointed out that the real problem wasn’t with Watson’s comments or any potential genetic differences that may or may not exist, but with the fact that the entire field of IQ testing is rather questionable.

In other words, while people were reacting strongly and vehemently to the words and the information, the underlying assumption Watson based his comments on—the fact that intelligence tests are valid—is wrong.

The “Booksthatmakeyoudumb” data suffers from the same problem: although it is controversial and interesting, the underlying assumption that a college’s favorite books, regardless of the caliber of their students, imply something about the intelligence of specific groups is highly questionable.

So, the next time you read something and feel yourself becoming angry, take a pause and see if the underlying assumptions truly make sense. It may just save you from embarrassment.

For the complete results, visit booksthatmakeyoudumb.virgil.gr.

Advertising