You hear this from age 12. It’s true, but it doesn’t tell you what to do. The point of this lecture: there is a real, formal sense in which we can sometimes recover causal effects from observational data — and a real, formal sense in which we usually can’t, no matter how big the dataset.
“I used to think correlation implied causation. Then I took a statistics class. Now I don’t.” — Sounds like the class helped. — “Well, maybe.” [xkcd 552]
A famous pedagogical paper (Matthews, Teaching Statistics, 2000, “Storks Deliver Babies, \(p = 0.008\)”) plots the number of storks against the number of babies across European countries, runs a regression, gets a significant \(p\)-value, and uses it to warn students that \(p\)-values can mislead. The headline is fine; the analysis is a mess, and it’s worth seeing why, because the same mistakes show up constantly.
Now flip it. Scrape all 50 US states for number of obstetricians and number of births (easy with a coding agent). Births vs. obstetricians is strongly correlated — obviously, via population: more people, more of both. Divide by population (better: by women aged 15–44, since age structure matters) and a robust trend survives, \(R^2 \approx 0.36\), tiny \(p\)-value, stable even if you drop the DC outlier. It is a real association.
Would you conclude “obstetricians deliver babies”? Of course not — any causal link is roundabout. But here’s the uncomfortable point: if the trend had come out flat, or slightly positive, nobody would blink. It would go straight into a textbook as a routine exercise. We scrutinise the stork claim because we know the answer in advance, and wave through the obstetrician claim for the same reason. That asymmetry means you can’t actually trust the causal stories attached to most regressions in most textbooks.
There’s a spectrum of responses, from strict to lenient:
The workable stance is epistemic humility: not nihilism (“believe nothing”), but awareness that every observational study — and to a degree every experiment — is bending the strict rules somewhere, so hold your conclusions loosely. What is not a matter of opinion, because the math guarantees it, is that trends which look completely real can arise with no causal link at all. The rest of these notes is about the cases where you can, carefully, do better than “there’s a correlation.”
The cleanest working definition (the counterfactual one):
\(A\) caused \(B\) if \(B\) would not have happened had \(A\) not happened.
This is intuitive, but it’s hard to apply: we never observe both worlds. For any given person who smoked and got cancer, we don’t see the version where they didn’t smoke. Causal inference is the project of extracting counterfactual statements from data we can see.
This lecture follows the framework that’s now standard in the field: causal models live on a directed acyclic graph (DAG), and the formal object we want is \(P(Y \mid \mathrm{do}(X = x))\), distinct from \(P(Y \mid X = x)\).
A directed acyclic graph (DAG) is a picture of how the variables in a system generate each other. Each node is a variable. Each arrow \(A \to B\) means “\(A\) is one of the things that determines \(B\)” — i.e., \(A\) enters into the mechanism that generates \(B\).
The canonical Shalizi example:
Smoking ──► Yellow teeth
│
└──► Tar in lungs ──┐
├──► Cancer
Asbestos ───────┘Reading this graph as a generative recipe:
Smoking (0 or 1) from some marginal distribution.Asbestos from its own marginal distribution, independently of Smoking.Yellow teeth and Tar in lungs from Smoking.Cancer from Tar in lungs and Asbestos.The graph encodes our assumptions about the data-generating process. Different DAGs make different predictions about which variables are (conditionally) independent.
Caveat. An arrow \(A \to B\) does not mean \(A\) always causes \(B\) on every instance — it means the mechanism that generates \(B\) takes \(A\) as an input. A drawn arrow says “\(A\) might affect \(B\)”; the absence of an arrow says “\(A\) definitely doesn’t (directly) affect \(B\).” Absences carry more force than presences.
Three patterns are worth memorising:
1. Common cause. Yellow teeth ← Smoking → Tar in lungs. Yellow teeth and Tar are not independent (smokers tend to have both), but they are independent conditional on Smoking. If you already know whether someone smoked, knowing their teeth tells you nothing extra about their lungs. Conditioning on a confounder blocks the path.
2. Chain. Smoking → Tar → Cancer. Smoking and Cancer are not independent, but they are independent given Tar (if the only way smoking causes cancer is via tar — strong assumption, but the point is structural).
3. Collider — “explaining away”. Tar → Cancer ← Asbestos. Marginally, Tar and Asbestos are independent (one is caused by smoking; the other isn’t). But conditional on Cancer, they become dependent: if someone has cancer and we learn they were exposed to asbestos, that “explains” the cancer, lowering the probability they also had tar buildup. Conditioning on a collider opens a path that was closed.
These rules together are called d-separation, and they’re the link between a DAG’s structure and the conditional-independence statements you can read off it.
A collider you’ve lived through. GPA → Admitted ← GRE. In the general applicant pool, GPA and GRE are positively correlated — people good at one tend to be good at the other. But look only at admitted students, who got in on roughly the sum of the two scores, and the correlation flips negative: a high GPA “explains” the admission, so the admitted high-GPA students tend to have lower GREs. This is the origin of the folk belief that people are either “math people” or “humanities people.” Inside a selective program — where you had to be strong at something to get in — being good at one subject really does predict being worse at the other. In the population at large it’s the opposite: skills go together. The apparent trade-off is an artefact of conditioning on the collider (admission).
set.seed(1)
n <- 50000
smoking <- rbinom(n, 1, 0.3)
asbestos <- rbinom(n, 1, 0.1) # independent of smoking
tar <- rbinom(n, 1, ifelse(smoking == 1, 0.85, 0.05))
yellow <- rbinom(n, 1, ifelse(smoking == 1, 0.60, 0.05))
p_cancer <- pmin(0.02 + 0.30 * tar + 0.50 * asbestos, 0.95)
cancer <- rbinom(n, 1, p_cancer)
df <- data.frame(smoking, asbestos, tar, yellow, cancer)Check the three patterns:
# 1. Common cause: yellow and tar are marginally dependent (both caused by smoking).
cor(df$yellow, df$tar)## [1] 0.4993546# But independent given smoking:
cor(df$yellow[df$smoking == 1], df$tar[df$smoking == 1])## [1] -0.002611089cor(df$yellow[df$smoking == 0], df$tar[df$smoking == 0])## [1] -0.002311224Marginally correlated (\(\approx 0.45\)); within strata of smoking, the correlation vanishes.
# 3. Collider: tar and asbestos are marginally independent.
cor(df$tar, df$asbestos)## [1] -0.002658525# But dependent conditional on cancer:
cor(df$tar[df$cancer == 1], df$asbestos[df$cancer == 1])## [1] -0.5038938Marginally near zero, conditionally negative. This is “explaining away” in action: among cancer patients, learning that someone was exposed to asbestos reduces the probability they had tar buildup, because the asbestos already accounts for their cancer.
The conditional dependence is a graphical fact, not a quirk of the simulation parameters. It would hold for any data generated by this DAG.
What is the causal effect of asbestos on yellow teeth?
By construction: zero. There is no path in the DAG from Asbestos to Yellow teeth. They’re causally unrelated.
A naïve analyst might run
summary(lm(yellow ~ asbestos + cancer, data = df))$coefficients## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.1876197 0.001984288 94.55264 0.000000e+00
## asbestos -0.1376235 0.006571463 -20.94260 5.727435e-97
## cancer 0.2634974 0.005386273 48.92017 0.000000e+00and find a “significant” coefficient on asbestos — because conditioning on cancer opens up the collider path. The coefficient is real (the conditional association is real), but it has no causal interpretation. “Adjusting for cancer” was exactly the wrong move.
The correct regression for the causal question “does asbestos affect yellow teeth?” is
summary(lm(yellow ~ asbestos, data = df))$coefficients## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.216416252 0.001939769 111.5680539 0.0000000
## asbestos -0.005154271 0.006129187 -0.8409388 0.4003862The coefficient on asbestos is essentially zero. Good — that matches the truth in the DAG.
This is the entire reason DAGs matter in practice. “Just throw everything into the regression” is not a safe default. Whether you should condition on a variable depends on its role in the causal graph: confounders should usually be in the regression; colliders and descendants of the treatment should usually be left out.
This is the heart of the matter. There are two very different questions you might ask:
The first is just conditioning on observed data. The second is asking about a hypothetical experiment.
These two are not equal in general. They’re only equal when \(X\) doesn’t share a cause with \(Y\) — i.e., when conditioning on \(X\) behaves the same as physically setting \(X\).
The cleanest way to internalize \(\mathrm{do}\) is to simulate from the modified graph. Setting \(\mathrm{do}(\text{Asbestos} = 1)\) means: re-run the data-generating process, but always force Asbestos to 1, ignoring its usual marginal distribution.
intervene_asbestos <- function(n, asbestos_value){
smoking <- rbinom(n, 1, 0.3)
asbestos <- rep(asbestos_value, n) # forced
tar <- rbinom(n, 1, ifelse(smoking == 1, 0.85, 0.05))
yellow <- rbinom(n, 1, ifelse(smoking == 1, 0.60, 0.05))
p_cancer <- pmin(0.02 + 0.30 * tar + 0.50 * asbestos, 0.95)
cancer <- rbinom(n, 1, p_cancer)
data.frame(smoking, asbestos, tar, yellow, cancer)
}
df_do1 <- intervene_asbestos(50000, 1)
df_do0 <- intervene_asbestos(50000, 0)
mean(df_do1$yellow)## [1] 0.21428mean(df_do0$yellow)## [1] 0.21334These two means are essentially identical: \(\mathrm{do}(\text{Asbestos} = 1)\) and \(\mathrm{do}(\text{Asbestos} = 0)\) produce the same distribution of yellow teeth, because asbestos has no causal path to yellow teeth in the DAG.
That’s the “no causal effect” claim formalised.
Meanwhile, the conditional probability among people who happened to have asbestos exposure and cancer was different from the marginal — that’s the observational lm(yellow ~ asbestos + cancer) finding from earlier. The two statements are not in conflict: one is about the joint distribution of observables; the other is about an intervention. They’re answering different questions.
The next question: when we don’t have the luxury of a randomised experiment (the actual gold standard for computing \(P(Y \mid \mathrm{do}(X))\)), can we recover causal effects from observational data alone?
The answer is sometimes, and three classical strategies cover most of practice.
If we have the DAG and observe enough variables, there’s a formal recipe (the backdoor criterion) for which set of variables we should condition on to make a regression coefficient have a causal interpretation. The rough rule:
This is the move that turns “\(X\) is associated with \(Y\) controlling for \(Z\)” into “\(X\) causes \(Y\).” But it requires the DAG: there’s no algorithm that picks the right controls from the data alone.
Suppose you want the effect of Smoking on Health, and you suspect there are unmeasured confounders (people who smoke might also live unhealthier lives in ways you can’t observe). An instrument \(I\) is a variable that:
Smoking,Health except through Smoking, andHealth.Cigarette tax is a candidate: it changes with state and over time, drives smoking rates, and arguably doesn’t affect health except through smoking.
Tax ───► Smoking ───► HealthIf those three conditions hold, you can read off the causal effect of Smoking → Health from the relationship between Tax and Health, even with unmeasured confounders on the smoking↔︎health link. This is a strong assumption — instrument-validity arguments are usually the hardest part of an IV paper.
For each treated unit (e.g., a person who smoked), find an untreated unit that looks the same on every measured covariate. Compare them. Repeat. The estimated effect is the average of the within-pair differences.
The hope: if you’ve matched on every confounder, the matched pairs differ only on the treatment, so you’ve recovered the counterfactual. The catch: you have to actually have measured the confounders, and you have to pair on all of them. Propensity score matching is the most common automated version.
Backdoor adjustment does sometimes pay off. In a data-science course, students could opt in to interacting with the teaching assistants (as part of a pedagogy study). Students who opted in did better in the course — but so what? Maybe keeners opt in, and keeners do better anyway. The naïve “opting in → higher grade” association is exactly the confounded kind.
The trick was to find a proxy for the confounder. The same course had an optional extra-credit assignment worth almost nothing — so doing it mostly signalled enthusiasm for the course. Draw the DAG:
Enthusiasm ──┬──► did extra credit
├──► signed up for the study
└──► test 1 / test 2 performanceEnthusiasm is unobserved, but “did the extra credit” is a measured child of it. Folding this structure into the data — conditioning on the extra-credit signal to partly account for enthusiasm — the effect of participation on first-test performance survived. That’s a case where thinking about the causal structure, and finding an observable proxy for the confounder, turned “just a correlation” into a defensible causal claim. It usually doesn’t work out this cleanly, which is exactly why it’s worth doing when it does.
A motivating worry: people who brush their teeth a lot also tend to exercise more, eat better, and so on, because they share a latent trait (“conscientiousness” / health-consciousness). A naïve regression of Heart Disease ~ Brushing will say brushing prevents heart disease — but the effect could be entirely driven by the confounder.
The DAG (Shalizi, Ch. 22):
Health-consciousness ──────────────┐
│ │
▼ ▼
Brushing ──► Gum disease ──► Inflammation ──► Heart disease
│ ▲
│ Exercise ──► (fat in diet) ─────────────┘
▼
(paths via the dental side)To estimate the causal effect of brushing on heart disease, you’d need to either (a) observe and condition on health-consciousness (impossible — it’s latent), (b) find an instrument that shifts brushing without affecting heart disease through any other channel (also hard), or (c) run an experiment where brushing is randomized (ethically and logistically dubious).
This is the honest reason most observational claims about lifestyle and chronic disease are heavily caveated. The DAG forces you to be explicit about the assumptions that would make a causal claim defensible — and often, no assumption available to you is defensible.