Where a Causal Model Stops

The Scope Problem

Consider a model the library might plausibly contain: a wage-elasticity model fit on local labor-market variation in low-wage retail and food-service sectors, using minimum-wage changes across US states 2005–2018 (the methodological lineage of Card and Krueger). The model identifies the causal effect of moderate minimum-wage increases on employment and prices. Internal validation: clean. Identification: holds under standard assumptions about parallel trends and limited spillover.

A user asks: what would happen to employment under a 50% national minimum-wage hike?

The library finds the model. The model has wage elasticity in its variable set. The query parameter is in the model's vocabulary. The library produces a number. The number reads as a clean policy estimate. It is also wrong — not because the math is wrong but because the regime the model implicitly assumes (small marginal hikes absorbed by employers without structural reorganization) does not extend to the magnitude being asked about. At a 50% national change, employers reorganize, automation thresholds get crossed, prices restructure, and general-equilibrium effects matter that a partial-equilibrium model cannot see. None of that was in the data the model was fit on.

Internal validation only checks the frame. It cannot tell you whether the frame applies. To detect out-of-scope errors, scope has to be declared as a first-class object in the library — and declared along enough dimensions that the things that actually go wrong can be caught. Which turns out to be more than population and time period.

What a Scope Declaration Contains

A scope declaration is the structured statement of the frame within which a model's causal claims hold. It has four fields. Each is independent: a model can be in-scope on one and out-of-scope on another, and the failure modes do not look alike.

1. Population scope

The most familiar dimension. Who is the model about — what demographic, geographic, sectoral, and temporal cohort does it claim to apply to? The wage-elasticity model's population scope: working-age adults in formal employment in low-wage retail and food-service sectors, US states, 2005–2018.

Population scope is the easiest to declare and the easiest to check automatically. It is also the dimension most likely to produce false confidence — a model whose population scope checks out on the surface (US, working-age, retail) might still fail on regime, measurement, or identification scope without any of the easy checks firing.

2. Regime scope

The structural and institutional context under which the model's causal claims hold. Distinct from population because the same population under different regimes obeys different causal structures.

The wage-elasticity model assumes a regime in which wage hikes are local and small relative to the wage distribution; firms cannot exit the market at the relevant timescale; substitution between labor and capital is limited at the margin; product markets are not contemporaneously restructuring. None of those assumptions appears in the variable set. They are background conditions under which the structural equations were estimated.

Regime scope is what breaks under structural change even when the population is identical. A 50% national hike does not change who is in the labor force; it changes the regime in which their labor is priced. A library that does not separately track regime scope from population scope will silently extrapolate across regime changes. The transportability literature (Pearl & Bareinboim, 2014) formalizes the conditions under which causal effects estimated in one setting carry to another; regime scope is one of the dimensions along which transport can fail.

3. Measurement scope

What was actually measured, with what instrument, under what sampling frame, with what missingness mechanism. The dimension that breaks when "income" in production data means something operationally different from "income" in the training data.

This is the connection back to referent declarations. Where a referent declares the quantity a variable denotes — unit, basis, population, time grain — measurement scope declares the operational definition under which the model's training data was collected: the survey instrument, the response rate, the kinds of non-response that were systematic, the proxies used when the underlying quantity was unobservable. A model trained on payroll-reported wages cannot be applied without translation to a population whose wages are largely tipped or informal.

4. Identification scope

The most subtle field, and the one that makes scope a per-query property rather than a per-model property.

A model can be in-scope on population, regime, and measurement, and still produce a wrong answer because the specific query being asked is not identifiable from the model's structure. A model that cleanly identifies P(employment | wage) from observational data does not, on the same data, identify P(employment | do(wage)) without further assumptions. Different rungs of Pearl's ladder require different identification assumptions; the same model can be in-scope for one rung and out-of-scope for another over the same variables.

Identification scope tags each model with the queries it can answer. Rung-1 only? Rung-2 under specific assumptions? Rung-3 with a fully specified structural model? The library refuses to answer queries outside the model's identification scope — even when population, regime, and measurement all check out. This is the field that ties scope most directly to Pearl's ladder: it makes the rung of a query a scope variable, not just a methodological label.

A Worked Example

Apply the four fields to the wage-elasticity case from §1. The model declares its scope along all four dimensions, in machine-readable form:

The user asks: what would happen to employment under a 50% national minimum-wage hike? The library checks all four fields before deciding whether to answer.

The model was not broken. The first version of the user's question was outside the model's frame on the regime dimension; the second version was inside the frame but exposed a measurement mismatch; the third version surfaced a per-rung identification check the user did not know to ask about. At every stage the library either answered with the active scope reported, or declined with a specific reason. The naive library — the one without scope as a first-class object — would have produced four numbers, all fluent, all wrong in different ways.

How Scope Flows Through the Library

Scope earns its place in the library only when it is more than metadata. A library that treats scope as documentation has not done the work; a library that treats scope as input to four operations has. Those operations are what turn declared scope into enforced scope.

Retrieval

A user query carries implicit scope: a target population, a time horizon, a regime context, a question type. The library matches query scope against candidate models' declared scopes during retrieval, not after. Models that are out-of-scope on the obvious dimensions never reach the candidate set. This is the cheapest scope check, and it is also the most underused — most retrieval systems match on topic, not on frame.

Composition

When two or more models are joined, the composed model's scope is the intersection of its parents' scopes. Scope intersection is one of the four composition primitives; this page gives it a foundation. The intersection can be empty, narrower than either parent, or further narrowed by identification incompatibilities introduced in the join. The data-fusion literature (Bareinboim & Pearl, 2016) develops the rigorous theory for fusing causal evidence across heterogeneous sources; the library's job is to expose the operational form. The library reports the resulting active scope before any composed query is answered.

Querying

Identification scope is per-query, so each ask is re-scoped at query time. A model in-scope on population, regime, and measurement can still be out-of-scope on identification for the specific question being asked. The library does not assume that a previously answered query of one rung implies anything about the next query of a different rung over the same model.

Reporting

The active scope is reported alongside any answer the library produces. Not in a footnote — inline, at the same level of prominence as the answer itself. A user reading the result can immediately see what was assumed, what was narrowed, and where the answer stops being valid. No scope is silently inherited; every answer carries its scope with it. The lineage here is the model-cards tradition (Mitchell et al., 2019), generalized from documentation that travels with a trained model to scope that the library actively enforces and reports per query.

What Scope Can't Do

Scope, like every other library primitive, addresses a category of failure rather than the whole problem. Four failure modes lie outside what scope checks can catch on their own. Pretending otherwise is its own dishonesty.

Misdeclaration

A modeler claims wider scope than the model warrants. Every downstream check trusts the declaration; if the declaration is wrong, the checks pass and the answers are wrong. Misdeclaration is the hardest failure mode because it sits upstream of the library entirely. Mitigations — cross-validation against held-out populations, peer review, scope audits, scope-narrowing priors — exist, but none are automatic, and the library cannot detect misdeclaration without external evidence.

Contestation

Two domain experts disagree on whether a model applies to a given query. One reads the regime as continuous with the model's training distribution; the other reads it as a structural break. The library can surface the disagreement and present both views as competing scope claims. It cannot adjudicate. This is where scope becomes scientific argument, and the library's job is to make the argument visible — not to resolve it pretending to be neutral.

Decay

Scope shrinks as the world changes. A model's measurement scope assumed iOS app-attribution worked; iOS 14 broke that. A model's regime scope assumed a stable interest-rate environment; the regime changed. Scope-decay primitives — automatic detection of when a model's scope has narrowed since training, and revocation of in-scope claims that no longer hold — are not standard library infrastructure. They should be. They are not yet.

Gaming

The LLM mediator paraphrases the user's question to fit the model's scope, defeating the check. The user asks about a 50% hike; the mediator silently reformulates the query as "small marginal hike, scaled" so the regime check passes. Every other primitive on this page assumes the query reaches the library faithfully. The mediator stands between the user and the library. Whether it tells the truth at that interface is a different problem.

Key Terms

↑ Back to top