Statistically meaningless.

The misapplication of statistics is a pet peeve for me. People throw stats around in ways that are totally meaningless. What that bugs me the most is when otherwise-reputable news sources calculate the odds of something non-random to demonstrate that it’s extremely unlikely.

For example, the odds of any one person in the United States being struck by lightning in a given year are 1/700,000. This is calculated by taking the total number of people struck by lightning in the U.S. each year and dividing it by the number of people in the U.S. The odds of any one person being struck in their lifetime are 1/3,000. This is based on assuming a lifespan of 80 years. Yet if I were to survey all of the life-long 80-year-olds residents of Manhattan, I suspect I’d find that significantly less than 1/3,000 had been struck. Why? Tall buildings — lightning doesn’t strike people in Manhattan. On the other hand, if I were to survey all of the life-long farmers in Nebraska, I imagine that I’d find a rate higher than 1/3,000, since they’re often the tallest thing in their fields, and are more likely to find themselves caught out in a storm.

It’s with this in mind that I say that I’m irritated by a recent story in Newsday, “Double dose of perfection: Twin brothers both get a flawless 1600 on the SAT, but Long Beach seniors simply take it in stride.” It’s a feel-good story, as you can imagine, but it contains this bit:

Of the 1.4 million high school seniors who took the test in 2004, only 939 scored a 1600, according to the College Board, which administers the test. With those numbers, the odds of any two people getting that score would be almost 1 in 2.3 million — and that doesn’t even take into account whether those two people are related, never mind twins.

That assumes, of course, that getting a 1600 is the luck of the draw, a sort of standardized testing by lottery. By that logic, the biggest moron and Dillon & Jesse Smith have the same odds of getting a 1600. That’s not the case. Dillon and Jesse are reported to be intelligent, hard-working kids who took SAT preparation seriously. They’re also said to be unusually fast-learning, and their extracurricular activities include debate club, foreign language honor society, and the trivia club. Finally, as twins, they’ve had similar upbringings and similar resources available to them, such that their odds of scoring closely on the SATs increase, not decrease, compared to two strangers.

What are the odds that a pair of intelligent, hardworking, self-motivated twins would get 1600s on the SATs? A lot less than 1/2,300,000. Probably something closer to the odds of being struck by lightning, which, as you now know, are a lot better than you thought.

Published by Waldo Jaquith

Waldo Jaquith (JAKE-with) is an open government technologist who lives near Char­lottes­­ville, VA, USA. more »