Survey says: Girls just want to have fun!

[tikistitch]

The blogosphere is in an uproar today over an especially stupid article that nonetheless merited page 1 treatment in the New York Times, Many Women at Elite Colleges Set Career Path to Motherhood. We refuse to pay for the online Times, so we only saw the article when the wonderful jmhm quoted it.

Now, given that Tikistitch is all about motherhood (as well as apple pie, though we find baseball a bit boring unless the infield is particularly cute), we were skeptical of its conclusions.

In particular, we found it interesting that the author claimed to have conducted a "survey" of female college students which purported to show those girls were all gonna give up that law firm for the dirty diapers. Here's why: we do Science. (We have a Ph.D. 'n stuff.) And sometimes, we've been known to design, administer and analyze surveys. To quote the preznit, "It's hard work!" You can't just pull questions out of your butt and expect to get results that are, well, ya know, scientific. Or valid. Or anything.

A couple hours ago, Atrios put up this great link to Gelf magazine, where they actually dug up the so-called "survey." And, it truly sucks. If we were still a mean old TA, we'd give the reporter an F and make her cry, hahahaha!

What's the matter? Well, you can read the Gelf article, but we only need to look at the very first question to tell that something is rotten in Denmark. Here 'tis:

When you have children, do you plan to stay at home with them or do you plan to continue working? Why?

Um, 'scuse us here, but where the fuck is your skip pattern???? Some folks, and I know this may come as a shock to a well-edumacated NY Times reporter, may not want to have children! Or, being 18 years old, may not know yet! Wooo.



But the problem is worse than that. What was the response rate for this survey? It's not stated anywhere in the article, and that's one of the most important questions you ask when evaluating surveys: who took 'em, and even MORE important, who didn't. Survey takers have an amazing love/hate relationship with non-responders. We spend many brain-hours and gray hairs positing reasons why so many people would NOT deign to take our lovely surveys.

Think of it this way: if you were a college girl, busy cramming for the biology final, and if you weren't planning to have kids, and you received a dippy 37-item survey in your email that didn't allow you to state that simple fact. What would you do? We would quickly hit the delete button. See? Instant non-responder! And guess what, the non-responder was much more likely to NOT ever take maternity leave. This is what we Scientist call, "bad." The author should be terribly worried about this, but evidently, she was much more interested in making her deadline and heading out to spot the next "trend."

Another really obvious flaw, which once again glares from the very first question: if you want people to really state what they think, you absolutely can not make it obvious what you want them to say. People have a tendency to be nice and answer what they think you want to hear, or what looks good. It's a tendency that even has a fancy name, "social desirability," and it's been studied and quantified and mulled over. That is, people who do real surveys mull over this stuff. So stating right off that they want to know what you're gonna do about work when you have children is kind of tipping your hand, dontcha think? Especially when there's not option about your partner/spouse taking parental leave. Even male people can take care of babies. It's true. We've seen it done.

We hates it when the media mis-represents science. But we also hates it when noodle-brains in the media use pseudo-scientific methodology to shore up faulty conclusions. NY Times, spend some money, and send your reporter off to get some remedial survey training. Otherwise, you'd be just as well studying chicken entrails to figure out your latest "trend."

Comments

[dellaluna.livejournal.com]

I does seem like everyone in the world has or wants a baby besides me at the moment, but with that thought out of the way...

NY Times, spend some money, and send your reporter off to get some remedial survey training. Otherwise, you'd be just as well studying chicken entrails to figure out your latest "trend."

I think they used that chicken entrail reporting method on the lead up stories they printed before the Iraq war to support Bush and Comapnies reasons for attacking. @_@

[kimako.livejournal.com]

hee, hee! that was me above, but I was using my other journal. I really need to condense my two livejournals because having two is driving me nutty. ^^;;

[tikistitch.livejournal.com]

Oh, that's cool, I've just been reading Dellaluna. ^_^ I'm addicted to No Reservations too. I love how they run the "offensive material" warning every 15 minutes, even though they've bleeped him too.

[kimako.livejournal.com]

No Reservations is a great show! I really like Anthony.^_^

I crosspost a lot of stuff but I mainly use my dellaluna journal for all my movie chatter (as you can probably tell I like watching lots of movies & writing about what I watch) and I try to post doll stuff in my kimako journal since a lot of my dellaluna 'friends" don't care about dolls or Japanese pop culture. ^^;

[mythosidhe.livejournal.com]

See, this is the kind of article that I read and feel that it rings false, but I can't make a constructive argument as to precisely why, because I don't have all that fancy schoolin' (yet). I love having smart friends who can explain why the reporters are idiots. Yep, its a good thing.

Also, I must say hooray for the people who could see that survey for the load of hooey that it is!

"It was relatively clear to me...that she was either unable to construct a suitable survey or had already decided what answers she wanted to receive and constructed her survey based on what questions would induce these responses."

Hope for humanity yet...

[tikistitch.livejournal.com]

A few years back, when I was still studying dietary behavior, we got contacted by a local TV journalist who wanted to interview us about something we'd reported at a conference. But, he got it in his head that our results were about dieting to lose weight. Naturally, that's something a lot of people find interesting. But, it was WRONG, so I corrected him immediately. He responded by asking me the exact same question, but phrased differently, about 6 or 7 times. I REFUSED to supply him with the quote he was obviously, and clumsily, looking for.

Then when the report was finally broadcast, the promo showed footage of a woman picking at a salad, and said it was about "the battle of the bulge." I weep.

[blythou.livejournal.com]

Being a journalist myself, I loathe lazy journalism more than anything in the world. I'm sick of those poorly or non researched stories, sick of the trend hopping, sick of pseudo-scientific babble that want to pass off as serious...

And I'm outraged that in their survey, they haen't even considered the option of not wanting children. How twisted is that ?! I know that right now, women are more and more pressured into having a family (and a bigger one than that, please), I know family is promoted as the last true value, blah-blah, but I know plenty of friends who, like me, don't want to breed.

Last week or so, when I asked a gyno about getting my tubes tied, she told me that, even though French law authorizes it for any woman over 30 (whether she's had kids or not), no doctor would agree to perform it, because I DON'T HAVE KIDS... Crazy, eh ?

[tikistitch.livejournal.com]

It was the "survey" bitlet that really did it in for me. If she'd just said, "I sent a list of questions to a group of students, and here are some of the responses," well, it would still be a dumb article, but at least the terminology would be correct.

Last week or so, when I asked a gyno about getting my tubes tied, she told me that, even though French law authorizes it for any woman over 30 (whether she's had kids or not), no doctor would agree to perform it, because I DON'T HAVE KIDS... Crazy, eh ?

Really? Now, I can see why that would be so in the USA: doctors would be worried about lawsuits from a childless person who changed her mind. And we have a rather vociferous group of people who are, to be frank, opposed birth control in any way, shape or form. But in Europe? That's surprising.

[kujachan.livejournal.com]

<--- will utterly refuse to spawn. Even when one of those poor teenage "mistakes" (i.e., freaking out because you don't know you're pregnant or not) came in the way, I considered giving it up for adoption. In my workforce (mental health) I see far too many hellions to even consider having my own.

[tikistitch.livejournal.com]

Toddlers are my peeps. If only they could find a way to keep them that age, I'd have one as a pet. ^_^

[mmymoon.livejournal.com]

OH GOD YES THANK YOU. This kind of thing infuriates me, and people don't understand why I walk around all day muttering under my breath.

I love toddlers, too! Although I'm sure as hell not planning on having any -- my brother is six, I've got altrustic family urges! Uh, yeah!

(Although if the depo every horribly failed and my womb bore fruit -- which is less likely than if I had my tubes tied, after my years of chemical sterilization, booyeah -- I'd probably keep the dern thing. IF it was after eight weeks/onset ofnbrainwaves/RU486 cutoff. And once it was weaned, at, you know, five years old, Cory could raise it. YAY!)

I have the BIGGEST rant about attitudes towards science coming up in my journal... gyah. Grad school is making me want to punch people in the face.

[tikistitch.livejournal.com]

A rant? A rant? My kingdom for a rant. I must read and deposit my two cents....

[buttercup87.livejournal.com]

I took a crappy survery once in high school. It was about drugs and teen sex and stuff like that. HOWEVER, I helped my Mom with her Stats class in grad school so I noticed that a lot of the questions were like the ones in this article. There were things like "When did you first have sexual intercourse (making love, going all the way, doing it)?". I was in NINTH grade at the time and I was really shy, so my eyes went O_O. Like I said, it was a crap survey with stupid questions. What they WANTED to know was if awareness of certain programs for adolescents made a difference in teen pregnancy, drug use, etc.

[tikistitch.livejournal.com]

Wow, that "survey" sounded like it was asking some *awfully* personal questions. Review boards get really touchy if you start asking people questions about sex (and drugs and rock 'n roll). I hope it was anonymous, right? They should've made a big deal about how they'd keep your answers confidential too.

And, yeah, if you're doing a real survey, you would start out asking something like, "Are you sexually active?" And then if you're not, you can skip all the personal questions.