Against Regression
For an Executive ReaderRegression estimates an association. It cannot tell you what would happen if you intervened, whether an outcome was caused by a specific factor, or what would have been different if the world had been different.
01 The claim
Regression answers an associational question. Most people use it to answer a causal one.
Regression is the right tool for one question: given that X and Y move together in the data, how much does Y tend to change when X changes by one unit? That is a useful question. It is not a causal question.
The problem is not regression. The problem is using regression to answer questions it was not designed to answer, and then acting on the answers as if they were causal. That happens constantly, in every industry that does quantitative analysis, because causal questions and associational questions look identical when written down.
The questions look the same. The answers are not. “What is the effect of X on Y?” sounds like a causal question. In regression it returns a conditional association. In a causal model it returns an interventional distribution. These are not the same number, and in the presence of confounding they are not even close.
02 What regression does
P(Y|X) is not P(Y|do(X)). They look the same. They are not.
Regression estimates P(Y | X), the distribution of Y in the population of cases where X takes a particular value. It is a Rung 1 operation on Pearl’s Ladder: it describes the world as it is observed, not as it would be under an intervention.
Rung 1: Association. P(Y | X = x). What is Y like in cases where X = x? This is what regression estimates. It is the right answer to a statistical question.
Rung 2: Intervention. P(Y | do(X = x)). What would Y be if we set X to x, regardless of what would normally cause X to be that value? This is what a decision-maker needs. Regression does not answer it.
Rung 3: Counterfactual. P(Yx | X = x′, Y = y′). What would Y have been for this specific case if X had been different? This is what a court, a regulator, or a post-mortem needs. Regression does not answer it.
The difference between Rung 1 and Rung 2 is confounding. When X is not randomly assigned, which is almost always, the cases where X = x are a selected subset of the population. Regression conditions on that selection. An intervention removes it.
03 What regression cannot do
Confounding, intervention, individual causation. Three failures, one root cause.
Three failures, each arising from the Rung 1 / Rung 2 gap:
Confounding. Regression observes that firms with higher training budgets have better outcomes. It cannot tell you whether training causes the improvement or whether better-run firms both train more and perform better. Without a structural model, there is no way to separate the two.
Intervention prediction. A marketing mix regression finds that TV spend has a coefficient of 0.8. It cannot tell you what would happen if you increased TV spend by $1M next quarter, because in the data, TV spend was high during periods when other things were also good. Do(TV = high) is not the same as obs(TV = high).
Individual causation. A regression can say that statins reduce cardiac events by 15% on average. It cannot say whether this patient’s cardiac event would have occurred had they taken the statin. The but-for question is a Rung 3 question. Regression operates at Rung 1.
None of these are failures of execution. They are failures of category. Regression is doing exactly what it was designed to do. The error is asking it to do something else.
04 The alternative
Encode what causes what. Then the data has something to inform.
The alternative is not to abandon statistical analysis. It is to encode the causal structure explicitly, as a directed acyclic graph with structural equations, and let the data inform the parameters within that structure.
The structural model encodes what causes what. Training budget and firm performance are both caused by management quality. Once that structure is explicit, the model can separate the effect of training from the selection effect. Regression cannot make that separation because it has no structure to separate along.
do() is the intervention operator. P(Y | do(X = x)) severs the incoming edges to X and asks what Y would be in a world where X was set to x externally. This is what a decision-maker controls. It is not estimable from a regression without additional structural assumptions, and those assumptions, if they exist, should be made explicit.
E[Y | X=x] # association: what Y looks like when X=x
# A causal model gives you this
E[Y | do(X=x)] # intervention: what Y would be if we set X=x
E[Y_x | X=x′, Y=y′] # counterfactual: what Y would have been for this case
# These are the same number only under randomisation
# In observational data they diverge whenever confounding is present
05 Where regression still fits
Prediction, description, screening. Rung 1 is not wrong, it’s just not enough.
Regression is the right tool when the question is genuinely Rung 1: what is the relationship between these variables in this population? Prediction, description, and screening are legitimate Rung 1 tasks.
Prediction. If you want to predict Y from X without caring why Y moves, fraud scoring, credit default prediction, readmission risk, regression and its descendants are appropriate. The goal is accuracy, not understanding.
Feature selection. Regression can identify which variables carry signal. That is a Rung 1 task. Whether those variables cause the outcome is a separate question that requires a causal model to answer.
Parameter estimation within a causal model. Once the causal structure is elicited from experts, regression can be used to estimate the parameters of the structural equations from data, within a model whose architecture is causally specified. Here regression is a tool inside a causal framework, not a substitute for one.
The error is not using regression. It is presenting regression coefficients as if they answer causal questions when the decision to be made is an intervention or a counterfactual.
06 In practice
Organisations optimise for what correlates with success. Not what drives it.
The practical consequence of the Rung 1 / Rung 2 gap is that organisations optimise for what the data shows correlates with success, rather than what actually drives it. Marketing budgets follow regression-derived ROI estimates that conflate brand equity with advertising effect. HR programmes are sized on regression coefficients that absorb selection into training. Supply chain investments are ranked by correlation with resilience, not causal contribution to it.
In each case, the right question is not hard to state: what would happen if we changed this? The causal model answers that question. The regression answers a different one and presents it in the same language.
The move is not to run the regression and then add caveats. It is to encode the causal structure first, elicited from the people who know how the business works, and use the data to estimate the parameters within that structure. The result is a model that can answer intervention questions, supports robustness analysis, and produces certified outputs an auditor can inspect.