Why a Causal Model
A provost or dean reviewing elective pathway data faces a structural measurement problem. Three tracks — quantitative methods, analytical reasoning, and writing-intensive humanities — each report mastery rates that vary substantially. The standard interpretation is that the higher-performing tracks are better preparation for Year 3 core courses. But Academic Preparation and Self-Direction both independently drive which elective a student chooses and whether they achieve mastery. Students with stronger prior preparation are more likely to select quantitative electives and more likely to achieve mastery regardless of which elective they complete. Observing the mastery rates of students who chose each pathway cannot separate the effect of the pathway from the effect of being the kind of student who chose it.
A causal model makes this separation explicit. Academic Preparation is a confounder: it has a direct effect on Elective Track selection and a separate direct effect on Mastery Rate. Any comparison of mastery rates across tracks that does not account for this confounder is measuring, at least in part, the prior preparation of the students — not the effect of the pathway. The gap between what the data shows and what a mandatory pathway recommendation would actually cause can be substantial.
| Analysis Component | Standard Approach | Causal Approach |
|---|---|---|
| Elective pathway comparison | Compare mastery rates across tracks; conclude higher-rate tracks are causally better | Intervention query severs Academic Preparation confounder; isolates the effect of the track from who selected it |
| Student-level counterfactual | Cannot ask: would this specific student have achieved mastery under a different pathway? | Anchors the student's actual preparation and self-direction via abduction; applies the hypothetical pathway change |
| Diagnostic root cause | Label underperforming tracks as low-quality; recommend discontinuation | Separates track quality from selection effect; identifies whether the track or the intake is driving the outcome |
| Policy recommendation | Mandate the highest observed-mastery track for all students | Quantifies the true causal effect of mandating; accounts for the fact that mandating changes who is in each track |
The Questions
- Holding everything about this student fixed, would switching from the writing-intensive humanities elective to the analytical reasoning elective before research methods have changed their mastery outcome? — Rung 3 (Counterfactual). Answering it requires the model to anchor this specific student's Academic Preparation and Self-Direction as observed background facts, then change only the elective choice and compute what mastery probability becomes; the confounder must be held fixed so it does not shift when the elective changes.
- If we recommend quantitative electives in Year 1 across all students, what does that actually cause to mastery rates in Year 3 core courses — separated from the Academic Preparation confounder that makes quantitatively stronger students more likely to choose those electives anyway? — Rung 2 (Intervention). A do() query severs the edge from Academic Preparation to Elective Track, isolating the causal effect of the track from the selection bias built into who currently chooses it.
- Three elective pathways show equivalent mastery rates on paper. Which pathway is genuinely producing the mastery, and which is populated by students who would have achieved mastery regardless of which elective they took? — Rung 1 (Association). The graph encodes which dependencies exist between Academic Preparation, Self-Direction, Elective Track, and Mastery Rate; entering each pathway as observed evidence propagates backward to the Academic Preparation distribution, revealing how much of the mastery signal is preparation rather than pathway.
Reading the screenshots: a black check mark on a node means it has been set as observed evidence — a fact entered into the model, acting as a filter. A red check mark means it has been set as a do intervention — a decision applied to the model, severing the influence of its parents.
Reading the spec tables: each Run the Analysis block lists the exact steps to reproduce each screenshot in Bayes Server. The Obs / Do column uses three italic control tokens: clear — reset the model to a blank no-evidence state; abduction step — enter the factual observations that anchor the U nodes to this specific case; use abduction result — apply a do() intervention with the U nodes held from the abduction step.
Would the elective choice have changed this student's mastery outcome?
“Holding everything about this student fixed — their academic background, their study habits, their program — would switching from the writing-intensive humanities elective to the analytical reasoning elective before research methods have changed their mastery outcome?”
The difficulty with this question in the data is that students who complete the humanities elective and students who complete the analytical reasoning elective are systematically different before they even enroll. The model anchors those background differences as fixed facts about this specific student, then asks what mastery probability would have been under the alternative elective. It is not asking what happens to a typical student — it is asking what would have happened to this one.
For a student with moderate Academic Preparation who completed the humanities elective and did not achieve mastery on the first attempt, the counterfactual probability of mastery under the analytical reasoning elective is 47.7% — compared to 41.6% on the actual pathway. The slideshow shows why AP staying fixed in that last step is the point, not a problem: slide 2 shows obs(Quantitative) pulling AP from 35.0% to 61.8% High through the back-door — that is the selection effect. Slide 3 shows do(Quantitative) holding AP at 35.0% — that is what a mandate would actually produce, a 5.5-point gap from the observed rate. Slides 4–5 then anchor a specific student at Moderate preparation and ask what their mastery would have been under a different track. AP stays at Moderate in slide 5 because do() changes the elective, not who the student was. The 6.1-point gain is the elective's genuine contribution for this preparation profile.
| Image | Obs / Do | Node | Set | Result |
|---|---|---|---|---|
| em-cf-0 | clear | — | Acad. Prep (High): 35.0%. Mastery: 48.3% Achieved / 29.7% Approaching / 22.1% Below | |
| em-cf-obs | clear | — | ||
| obs | Elective Track | Quantitative | Acad. Prep (High): 61.8% — selection inferred up. Mastery: 56.4% Achieved | |
| em-cf-do | clear | — | ||
| do | Elective Track | Quantitative | Acad. Prep (High): 35.0% — stays at prior. Mastery: 50.9% Achieved | |
| em-cf-1 | abduction step | — | ||
| clear | — | |||
| obs | Academic Preparation | Moderate | ||
| obs | Elective Track | Humanities | Mastery: 41.6% Achieved / 32.6% Approaching / 25.8% Below | |
| em-cf-2 | use abduction result | — | ||
| do | Elective Track | Analytical Reasoning | Acad. Prep: stays Moderate. Mastery: 47.7% Achieved / 30.4% Approaching / 21.9% Below |
All nodes at prior. Mastery Achieved 45%, Approaching 35%, Below Threshold 20%. Academic Preparation and Self-Direction at population base rates.
What does recommending quantitative electives actually cause to mastery rates?
“If we recommend quantitative electives in Year 1 across all students, what does that actually do to Year 3 mastery rates — separated from the fact that students who already choose quantitative electives tend to arrive better prepared?”
Academic Preparation drives both the elective choice and the mastery outcome independently. Students with stronger prior quantitative backgrounds are more likely to self-select into quantitative electives and more likely to achieve mastery regardless of pathway. When the provost compares observed mastery rates across tracks, the quantitative track appears more effective — but part of that signal is the preparation effect, not the track effect. The model removes this by treating the recommendation as a decision that is imposed rather than chosen, so the preparation distribution stays at the population mix rather than shifting toward the already-prepared.
Mandating quantitative electives raises mastery rates from 55.2% to 57.9% — a genuine 2.7-point causal gain. The observed rate for students who currently choose quantitative electives is 64.9%, which overstates the mandate's effect by 7.0 points. The gap reflects selection on two confounders simultaneously: Academic Preparation rises from 35.0% to 61.2% High and Self-Direction rises from 30.0% to 41.4% High when the Quantitative track is observed. Both shift back to the population mix under do(). For students with low Academic Preparation, the quantitative elective does not reach the mastery rates the observed data implies — the mandate helps the median student but the headline figure is driven by the intake, not the track.
| Image | Obs / Do | Node | Set | Result |
|---|---|---|---|---|
| em-int-0 | clear | — | AP (High): 35.0%. SD (High): 30.0%. Mastery: 55.2% Achieved / 25.3% Approaching / 19.4% Below | |
| em-int-1 | clear | — | ||
| obs | Elective Track | Quantitative | AP (High): 61.2% -- infers up. SD (High): 41.4% -- infers up. Mastery: 64.9% Achieved | |
| em-int-2 | clear | — | ||
| do | Elective Track | Quantitative | AP (High): 35.0% -- stays at prior. SD (High): 30.0% -- stays at prior. Mastery: 57.9% Achieved |
All nodes at prior. Mastery Achieved 45%. Academic Preparation at population mix: 35% High, 45% Moderate, 20% Low. No elective recommendation applied.
Which pathway is genuinely producing mastery, and which is a selection effect?
“Three elective pathways show equivalent mastery rates on paper. Which pathway is genuinely producing the mastery, and which is populated by students who would have achieved mastery regardless of which elective they took?”
At Rung 1 the model runs as a filter in both directions. Forward: enter the elective track as observed evidence and read the mastery distribution. Backward: enter the elective track and read what the Academic Preparation distribution infers to, revealing how much of the pathway's apparent mastery rate reflects the preparation of students who chose it rather than what the pathway adds. A pathway where entering the track shifts Academic Preparation strongly upward is one that is predominantly benefiting from selection, not from instruction quality.
Entering the Quantitative track as observed evidence shifts Academic Preparation from 35.0% to 61.2% High and Self-Direction from 30.0% to 41.4% High — both confounders shift strongly, confirming the mastery signal in that track is predominantly selection. The Analytical Reasoning track infers Academic Preparation to only 33.2% High and Self-Direction to 29.9% — both essentially at the population mix — while producing 59.3% mastery. The Humanities track reveals the starkest intake effect: Academic Preparation falls to 12.1% High and Self-Direction to 19.4%, well below population base rates. At 38.9% mastery from that intake, the track is not failing its students — it is serving the students who need the most support. The policy implication is not that quantitative is better: the Analytical Reasoning track is producing mastery from a representative intake, while the Quantitative track is benefiting from a self-selected cohort that would have achieved mastery on almost any pathway.
| Image | Obs / Do | Node | Set | Result |
|---|---|---|---|---|
| em-diag-0 | clear | — | AP (High): 35.0%. SD (High): 30.0%. Mastery: 55.4% Achieved / 29.7% Approaching / 14.9% Below | |
| em-diag-1 | clear | — | ||
| obs | Elective Track | Quantitative | AP (High): 61.2%. SD (High): 41.4%. Mastery: 69.7% Achieved | |
| em-diag-2 | clear | — | ||
| obs | Elective Track | Analytical Reasoning | AP (High): 33.2%. SD (High): 29.9%. Mastery: 59.3% Achieved | |
| em-diag-3 | clear | — | ||
| obs | Elective Track | Humanities | AP (High): 12.1%. SD (High): 19.4%. Mastery: 38.9% Achieved |
All nodes at prior. No elective track entered. Academic Preparation at population mix. Enter a track to see how much of its mastery signal reflects selection versus pathway contribution.
Download the Models
All models require Bayes Server (free edition available). See Download Models for the full library.
If your elective pathway recommendations are based on observed mastery rates, you are recommending the track that attracts the most prepared students — not the track that produces the most mastery. The policy will look successful in the data and fail the students who needed it most.
The models are free. What I provide is the judgment to build the right structure for your specific institution, encode your faculty’s knowledge into it, and turn the output into advising decisions your academic committee can act on. The discipline stays with your team.