BONN, Germany – For the past week or so, there’s been a certain amount of buzz about a new study that claims to show a causal connection between the consumption of internet porn and a decline in marriage rates. Does the data support the claim?

Filled with caveats and qualifiers largely lost in the retelling by the media outlets that have covered it, “Are Pornography and Marriage Substitutes for Young Men?” really amounts to a highly confounding lump of statistical guesswork composed by Michael Malcolm from West Chester University of Pennsylvania and George Naufal of Timberlake Consultants.

“Substitutes for marital sexual gratification may impact the decision to marry,” the abstract begins, reasonably enough—a measured tone which soon gives way to an overly optimistic assessment study’s embedded assumptions, unfortunately.

“Proliferation of the internet has made pornography an increasingly low-cost substitute,” the abstract continues. “We investigate the effect of internet usage, and of pornography consumption specifically, on the marital status of young men. We show that increased internet usage is negatively associated with marriage formation. Pornography consumption specifically has an even stronger effect. Instrumental variables and a number of robustness checks suggest that the effect is causal.”

Relying on data from the General Social Survey, which has been conducted by the National Opinion Research Center at the University of Chicago since 1972, the researchers report the rate of marital formation dropped by 39 percent between 1950 and 2010, with a 17-percent drop between 2000 and 2010.

After discussing marriage rate trends in greater detail, the authors state the “instrumental variables and a number of robustness checks” used in their analysis all “suggest that this is a causal effect and not merely the endogenous relationship that married men are less likely to look at pornography, or some kind of unobserved selection issue that distinguishes men who use pornography from men who do not use pornography.”

Therefore, the authors continue, “We assert that increasing ease of accessing pornography is an important factor underlying the decline in marriage formation and stability.”

That all sounds impressive, even downright science-y, until you take a closer look at the specific instrumental variables the authors used in the study.

If you’re not familiar with the term, an “instrumental variable” is a device used by researchers to estimate and assess potential causal relationships when a controlled experiment or other means of confirming causality isn’t available, or simply isn’t feasible. They’re useful tools for statisticians and researchers, to be sure, but it’s also crucial to remember instrumental variables are only as solid and reliable as the researchers who use them.

A good instrumental variable, the kind you can actually use to strengthen an observed correlation into a claim of causality, only impacts the precise outcome in which you’re interested (in this case a decline in marriage rates) through the particular means you hypothesize (i.e., watching too much porn on the internet). If there’s another means by which the chosen variable can impact the final result, then it’s simply not a good tool for the job.

In this paper, the two instrument variables chosen by the authors are the father’s level of education and the question of whether the respondent lived in an urban or rural area. Evidently, the first instrument was chosen because the authors reasoned the children of educated fathers would be more tech-savvy and more likely to be online. Similarly, the authors figured city dwellers are more likely to be watching porn online due to the superior internet infrastructure in urban areas.

Whatever the reason(s) for their selection or the strength of the assumptions underlying the choice, as noted by Slate’s Jordan Weisserman, both criteria are terrible instrumental variables in the context of this study.

To begin with, there are many ways in which having an educated father might affect your decision to get married. As Weisserman observes, your educated father might encourage you to put off getting married until after you have finished college, or until you have started your career. Among other things, this delay in getting married might increase the odds you’ve responded to something like the GSS survey before even considering the question of marriage.

Whatever the reason for putting off marriage, obviously there are many ways one could land in the “unmarried” portion of the dataset for reasons having nothing whatsoever to do with porn, which brings us back to the dataset itself. As good as the GSS data is (and it has provided a solid basis for a great deal of solid academic and scientific research over the years), it has its limitations. Among the more obvious limitations in the context of this study is the GSS doesn’t track its respondents over time.

In other words, if an unmarried 26-year-old urban dweller who is the son of an educated father responds to the GSS today by saying he’s unmarried and doesn’t plan to marry in the future, but then changes his mind two years later and gets married, the GSS data won’t reflect his eventual marriage. As a result, so far as Malcolm and Naufal’s paper is concerned, this hypothetical respondent would be in the “unmarried” camp forever, regardless his actual status.

There are other problems with the paper, of course, including the fact it only surveys men, as though the desires, expectations, opinions and actions of young, unmarried women have no impact on the rate at all. Apparently, all unmarried women are just waiting for some fellow to come along and slip a ring on their finger—you know, provided he’s not so addicted to internet porn he can’t find Jared’s on a map anymore.

As I’ve written before, the adult industry should be open to (and on the lookout for) good science on the subject of porn, even if the conclusions aren’t necessarily something we want to hear. Whatever this paper by Malcolm and Naufal’s might be, though, it is decidedly not “good science.”