Friday, December 17, 2010
Taken for a Spin
Oh, look: another study that proves that if you do X, your kids will thrive, but if you do Y, you've ruined them. Or did the study actually prove that? Katy Read looks at causation, correlation, and mom blame.
Subscribe to:
Post Comments (Atom)
Posts
9 comments:
A nicely written article explaining the perils and promise of correlation. Nonetheless, the downside to writing a lengthy article on errors in scientific reasoning no doubt leads to similar errors.
For example the author states:
"Keeping tabs on our kids may or may not be a surefire way to prevent them from binge drinking. (Hint: It’s not. Remember my old classmate whose dad would tail her with the headlights off? Let’s just say she managed to get her cup filled at the keg nonetheless.) "
While a one shot case study may provide fodder for future research and open the doors of dialogue, they are not a valid substitute for a well designed study. One can actually make less inferences from the experiences of the author's friend than they can from the much maligned BYU study.
-Mark
Thanks for reading, Mark, and you make an excellent point: Anecdotal evidence doesn't count for much. Though to clarify in this case, the key word is "surefire." Keeping tabs on kids may or may not reduce binge drinking, but one example is all you need to show the strategy doesn't infallibly prevent it. To be fair, the BYU researchers didn't claim it was.
-- Katy
*does, I mean.
(Speaking of errors!)
Very nice article! I laughed. I thought. I reduced my "TV is bad" guilt-o-meter a tad.
Nice exploration of the correlation does not equal causation problem...I love that you tracked down the BYU researcher and asked him why he would make a causal link statement in the press release.
There is an important discussion in the philosophy of science and the social sciences about whether we can ever establish this thing called causation. You should check it out before taking folks to task over their research that is, typically, designed to help parents rather than make them feel guilty.
As a researcher in maternal and child health (and a mom and BC reader), I greatly appreciate this article and I hope it helps readers to interpret social science research. A couple of other common errors you might want to think about are:
-Confusing statistical significance and clinical significance. You might read, and it might be true, that a study found that the IQs of breastfed babies are a point or two higher than those of formula-fed babies. Even aside from the confounding factor of the IQs of the breastfeeding mothers, not to mention the validity of the IQ as a measure of anything, who cares? The fact that the difference was statistically significant (i.e., big enough to report) just means that the probability that the difference seen in the study sample actually reflects a difference in the population as a whole is greater than 95%. Whether one IQ point (or one inch of height or any other measure of anything) matters to anyone is a judgement call, and no statistical test can measure it.
-Exaggerating the difference between small numbers. There were big stories a few years ago about a study that found that toddlers in daycare were THREE TIMES MORE LIKELY than kids who stayed home with their parents to score high on an aggression scale. As I'm sure you realize, there are all kinds of possible reasons for this: maybe the parents of aggressive kids think they need more exposure to other kids so they are more likely to put them in group care, or maybe the aggressive kids are more wearing on the parents so they look for outside care, or whatever. But the real problem was the actual numbers: 18% of the daycare kids were aggressive, compared to 6% of the kids who stayed home. So while yes, 18 is 3 times 6, the real story to me is that the vast majority of kids were not found to be aggressive, regardless of where they spent their days.
I'll tell you about the Ecologic Fallacy some other time.
@renee, interesting post and excellent points. And thanks for the tip about the Ecologic Fallacy. If you (or anyone) would be interested in further discussion of these issues, please click on my name to get my website, where I have an email link. I'd love to hear from you.
Thanks for the comments, everyone!
@ Anonymous, 5:53, I think she was more speaking to the media's interpretation of the data and its use of said data to generate dramatic headlines, which are often guilt-inducing.
@ Katy, thank you so much. I knew these things, but also was unable to shake the guilt. Your article somehow makes it easier for me to let go, at least sometimes.
Elizabeth
Salvation! Despite all the nitpicking, the point is that we all need to take a chill pill. Thank you for the reminder to listen more carefully to the perky news reporter's headline.
Say hi to Mpls for me - I recently moved away and I'm missing it terribly!
Post a Comment