Study: Surveys a Lousy Means of Determining Causality
NEW YORK – A groundbreaking new study conducted by researchers at South Bronx University’s Postal School of Public Health has documented a notable increase in the number of scientists who think surveys are a lousy means of determining causality.
According to the study’s abstract, “People who think you can determine a causal connection between things like condom use in adult films and condom use among viewers by asking 265 not-so-randomly selected gay men to self-assess their own habits and proclivities and how those things are influenced by sexually-explicit material clearly have too much confidence in the methods employed by the social sciences.”
Dr. Bart Flimshaw, head of the SBU research team, said another problem plaguing survey-based research comes not from those who conduct the surveys but from those who “over-sell the results,” often in furtherance of a socio-political agenda of some kind that has little to do with the science at hand.
“When the published research itself concedes ‘despite the addition of questions to assess the perceived influence of SEM [sexually explicit media] on sexual fantasies, desires and behaviors, the study was cross-sectional and therefore cannot determine the causal directionality between SEM and condom use,’ but people promoting the same research say it ‘documented a notable increase in condom use among viewers of adult films in which the performers wore condoms,’ someone is fudging things more than just a little bit,” Flimshaw said. “And he’s probably not one of the guys wearing a lab coat.”
At the same time he’s skeptical of surveys as a research tool, Flimshaw had to concede his team’s study, too, was based on a survey.
“Over the course of several days, between the hours of 2 a.m. and 4:30 a.m., we phoned over 400 physicists, biologists, chemists and other practitioners of various ‘hard sciences’ and asked them a series of carefully crafted questions about the use of surveys to assess and assign causality,” Flimshaw said. “After being hung up on by 131 of them and eliminating four others because they failed to meet our selection criteria of being able to successfully operate a telephone in the middle of the night without cursing profusely, we had our respondent pool of 265 scientists. If nothing else, we established a very high percentage of scientists find it extremely irritating to receive phone calls between 2 and 4:30 am.”
Flimshaw’s colleague, Dr. Gustav Utternahnzence, said the most interesting connection uncovered by the study, at least for him, was a strong correlation between being a practitioner of a hard science and thinking social scientists are “basically a bunch of glorified graduate students busying themselves with metaphorically plumbing their own assholes with their thumbs.”
Utternahnzence stressed this opinion was not among answers offered in the survey’s multiple-choice responses, but based entirely on outside-the-survey feedback offered directly by the respondents themselves.
“One respondent suggested I find a new job that offers a more rigorous intellectual challenge, like teaching kindergarten or working for Donald Trump’s campaign,” Utternahnzence said. “I admit the Trump idea is intriguing, but clearly I’m not qualified to explore the other possibility.”
Dr. Marvin Chomansky, an astronomer with the Chuck Norris Center for Space and Other Science Stuff in Houston, said the only surveys he trusts are the ones used to inform the questions on Family Feud.
“Obviously, the quality of the data has declined from the high water mark of having the questions and answers delivered by a thoroughly sloshed Richard Dawson, but when Family Feud says the number one thing people yell at their dogs is the word ‘no,’ I believe them,” Chomansky said. “But have a sociologist ask the same question, and somehow the answer turns into ‘condoms.’ Who the hell yells at their dogs about condoms, and more important, in what sort of context is this happening? America and all her dogs deserve an answer.”
Flimshaw said while he’s not sure why people would yell “condom” at their dogs, he still believes his team’s methodology and the work of other, similar social-science research teams is superior to that of Family Feud.
“The guy who hosts the show can’t even read the right beauty queen’s name off a card right in front of his face, but I’m supposed to trust what he says about people and dogs he’s never met?” Flimshaw said. “I’ll get my information about people and their dogs from properly vetted and professionally qualified sociologists, thanks — even when they’re from some obscure college in Columbia.”