Reasoning About Counterfactuals
“What would have happened if X had been different?” is one of the most powerful questions in analysis — and one of the most easily abused.
A counterfactual is a claim about what would have happened in a situation that did not actually occur. “If we had invested in solar earlier, we would be ahead of the curve today.” “If the regulation had not passed, the industry would have collapsed.” These are counterfactual claims, and they appear constantly in causal reasoning, historical analysis, and policy evaluation.
Why we need them
Counterfactuals are essential to causal reasoning. To say that X caused Y is, at least implicitly, to say that without X, Y would not have happened. The counterfactual “without X, no Y” is doing the causal work. Without counterfactuals, we cannot distinguish causation from mere correlation.
This is why counterfactuals are unavoidable in policy analysis. To say that a policy worked is to say that without the policy, the outcome would have been worse. The counterfactual is the comparison point. There is no other way to know whether a policy was effective.
Where they go wrong
Counterfactuals are hard because they describe situations that didn’t happen. We can’t directly observe what would have happened in the counterfactual world. We have to reason our way to it, using what we know about how the world works.
Several errors are common.
The first is treating one counterfactual scenario as the obvious alternative. When we say “without this regulation, the industry would have collapsed,” we are implicitly fixing one alternative scenario (no regulation, otherwise everything the same). But the actual alternative might have been “no regulation but different industry self-policing,” or “different regulation, less aggressive,” or any number of other configurations. Treating the “everything else equal” counterfactual as the relevant comparison is often misleading.
The second is using counterfactuals to justify a conclusion already held. “If they had done what I recommended, things would have turned out fine.” This counterfactual is rarely tested. It is a comfortable assumption that supports the speaker’s preferred reading of events. The hard work would be to construct the counterfactual carefully and ask whether the recommended action would actually have produced the predicted outcome.
The third is forgetting that counterfactuals get less reliable the more they diverge from actual events. “If we had nudged this slightly differently, this small change would have followed” is usually reasonable. “If history had taken a different path two centuries ago, the world today would look like X” is much less reliable. Each step of divergence introduces uncertainty, and the uncertainty compounds.
How to reason about them well
A few practices help.
Specify the counterfactual precisely. “Without this policy” is too vague. “Without this policy but with the alternative policy that was actually under consideration” is more useful.
Identify what the counterfactual relies on. Most counterfactual claims rely on background assumptions about how the world works. Make those assumptions explicit. Are they well-supported?
Consider multiple counterfactuals. Don’t fix on one comparison scenario. Consider several. If different reasonable counterfactuals produce different conclusions about the effect being studied, your confidence in any single conclusion should be modest.
Distinguish between counterfactuals that are testable and ones that aren’t. Some counterfactual claims can be checked against similar real-world cases. Others cannot. The first kind deserves more confidence than the second.
An everyday application
Most regrets are counterfactual claims. “If I had taken that job, I would be happier now.” The counterfactual is doing the regretting. But the counterfactual is also speculative. Many factors would have been different in the alternative life; assuming the rest would have been the same and only the job different is rarely accurate.
Recognizing this does not eliminate regret, but it does soften it. The counterfactual is less reliable than it feels. The alternative life is not knowable from where we stand. We can speculate, but we should hold the speculation lightly.
