The standard response to expert disagreement — average, defer, or iterate until consensus — produces a point estimate that conceals genuine uncertainty. A causal model encodes the disagreement as a probability distribution, which propagates honestly into every calculation the model makes and makes the uncertainty visible in the decisions it informs.
Disagreement Is Information
When two experts assign different probabilities to the same causal claim — one believes the failure probability is 15%, the other believes it is 35% — the temptation is to average to 25% and move on. The average is not the right answer. It discards the information that the two experts hold genuinely different beliefs, which is itself diagnostic.
The right question is: why do they disagree? Expert disagreement on a causal claim has four possible sources, and each has a different implication for what to do next:
Different evidence bases
Expert A has observed 200 cases of this failure mode; Expert B has observed 20. Their priors reflect genuinely different amounts of experience. The resolution is not averaging — it is eliciting what each expert has observed and constructing a shared posterior that conditions on both evidence bases. The expert with 200 observations should carry more weight, and the model can formalize exactly how much more.
→ Implication: the wider prior (Expert B) should update toward Expert A's posterior once the evidence differential is made explicit. If it does not, something other than evidence is driving the disagreement.
Different domain knowledge
Expert A specialises in the mechanism; Expert B specialises in the context. They are not disagreeing about the same thing — they are providing complementary information about different parts of the causal chain. Expert A knows how the failure mechanism operates; Expert B knows the environmental conditions under which it is triggered. The resolution is not choosing between them but integrating both into the model structure.
→ Implication: the causal graph may need to be extended — Expert B's contextual knowledge adds nodes that Expert A's mechanism knowledge alone would not have included. Disagreement about probabilities may dissolve once the model structure is richer.
Genuine irreducible uncertainty
Both experts have seen the same evidence and understand the same mechanisms, and they still disagree. This is the correct signal that the true probability is unknown — that the causal relationship is not well-established even among experts who have studied it. The resolution is a wide prior that honestly represents this uncertainty, rather than a false precision that pretends the disagreement has been resolved.
→ Implication: genuine expert disagreement produces a diffuse conditional probability distribution, which correctly propagates as high uncertainty through the causal model. High uncertainty on a key variable is high Value of Information — it is the signal that additional data would most improve the decision.
Model misspecification
Experts consistently disagree with each other — or consistently disagree with the model's outputs — because the causal graph does not include a variable that both experts know matters. The disagreement is a symptom of a missing node, not of uncertainty about a correctly-specified relationship. The resolution is not eliciting probabilities more carefully; it is expanding the model structure.
→ Implication: persistent expert-model divergence is the primary diagnostic signal for model misspecification. The expert who consistently finds the model's outputs implausible is providing information about what is missing from the graph, not evidence that their priors are miscalibrated.
How the Causal Model Handles It
The standard risk process treats expert judgment as a point estimate: the expert says “likelihood is 3 out of 5” and the register records it as such. One expert, one number, no representation of uncertainty. If a second expert says the likelihood is 2 out of 5, the process has no mechanism for handling the disagreement except to choose one number or average them.
A causal model handles expert disagreement at three levels:
Graph structure disagreement
If experts disagree about whether variable A causes variable B — whether the arrow should exist at all — the model building session is itself the resolution mechanism. The experts are asked to defend their claim: “what mechanism connects A to B? What would have to be true for A to affect B? Has this been observed?” This is not adjudication by seniority. It is a structured causal reasoning exercise that either produces consensus on the mechanism or identifies the specific empirical question that, if answered, would resolve the disagreement. That question has a VOI, which tells the organization whether to resolve it before building the model or treat the arrow as uncertain and proceed.
Conditional probability disagreement
If experts agree that A causes B but disagree about the magnitude — Expert A believes P(B | A=high) = 0.7; Expert B believes it is 0.4 — the model can encode both. The prior on the conditional probability table is itself uncertain. The model runs with a distribution over the parameter, propagating the uncertainty through every downstream calculation. The output is not a point prediction — it is a probability distribution that is wider precisely because the experts disagree. This is honest. It is also the input to the VOI calculation: the wider the distribution over the parameter, the more a targeted data collection exercise would be worth.
Delphi methods — iterative anonymous elicitation with feedback — are designed to produce consensus. They are effective at doing so. The problem is that consensus is not the goal. Accuracy is the goal. Expert consensus produced by a process that feeds experts each other's estimates tends to produce anchoring: later estimates converge toward earlier ones, not because evidence has been exchanged but because social information has. The causal model elicitation process is designed to exchange causal reasoning, not estimates — which produces genuine belief revision rather than social convergence to a defensible middle number.
Persistent divergence as a structural signal
When an expert consistently finds the model's outputs implausible — the model says 12% failure probability and the expert says it should be 40% — and the disagreement persists after the probability tables have been reviewed, the expert is almost always pointing at a missing variable. The correct response is to ask: “what do you know that the model doesn't? What factor, not currently in the graph, would explain why your estimate is higher?” The answer typically produces a new node that resolves the divergence — not by changing the expert's estimate but by making the model rich enough to agree with it for the right reasons.
When Disagreement Should Change the Decision
The connection between expert disagreement and decision-making runs through the Value of Information. The formal relationship:
- Expert disagreement on a causal parameter → wide prior on that parameter → high variance in the probability distribution over outcomes → high sensitivity of the optimal decision to the true value of the parameter → high VOI for evidence that would resolve the parameter's uncertainty
- Expert consensus on a causal parameter → tight prior → low variance → low decision sensitivity → low VOI → act on current knowledge
This gives the organization a formal decision rule for what to do with expert disagreement: compute the VOI of resolving the disagreement before acting. If the VOI exceeds the cost of the investigation that would resolve it, resolve it first. If the VOI is less than the cost, act on the current diffuse prior — the disagreement is not decision-relevant at the current stakes.
| Expert disagreement about… | Effect on model | VOI implication | Decision implication |
|---|---|---|---|
| A high-stakes causal link | Wide conditional probability table for a key node | High — resolving the disagreement would substantially change the expected-value calculation | Gather targeted evidence before committing to the high-stakes action |
| A low-stakes causal link | Wide conditional probability table for a peripheral node | Low — the peripheral node's uncertainty does not propagate meaningfully to the decision | Act on current knowledge; the disagreement is not decision-relevant |
| The causal structure itself | Model misspecification risk — a node may be missing | Potentially very high — a missing causal path can produce systematically wrong decisions | Resolve the structural question first; do not proceed with a model both experts find incomplete |
| The baseline rate of a risk | Wide prior on the root node of the causal chain | Moderate — depends on how sensitive the decision is to the baseline rate vs other parameters | Run sensitivity analysis to determine whether the decision changes across the range of expert estimates; act if robust, investigate if not |
The Elicitation Process in Practice
Separate structure from magnitude
Start with the causal graph — which variables cause which — before discussing any numbers. Structure disagreements are more fundamental than magnitude disagreements and should be resolved first. An expert who disagrees about whether an arrow should exist is making a different claim than one who disagrees about its strength. Conflating the two produces unresolvable sessions.
Ask for the mechanism, not the number
For each causal claim, ask the expert to describe the mechanism: “how does A cause B? Through what process? Under what conditions?” Mechanism descriptions reveal whether apparent disagreements are actually about different components of a more complex causal chain — in which case both experts are right about different parts of it. Mechanism descriptions also reveal when a number is genuinely uncertain: if an expert cannot describe the mechanism clearly, their probability estimate is not well-grounded and should carry a wide uncertainty interval.
Encode the disagreement, do not resolve it artificially
Where experts genuinely disagree after mechanism discussion, encode the disagreement as a wide conditional probability distribution. Do not average to a point estimate. Do not defer to seniority. The wide distribution is the correct representation of the current state of knowledge — and it propagates honestly into the model's outputs, making the uncertainty visible in the decisions it informs rather than hiding it in a false precision.
Identify what evidence would resolve it
For each genuine disagreement, ask: what observation would change your probability estimate? The answer is the design specification for the evidence that would resolve the disagreement. Combined with a cost estimate for obtaining that evidence, it is a VOI calculation — which tells the organization whether to gather the evidence before using the model for high-stakes decisions.
Your domain experts hold different views on the probability of your highest-stakes risks. Those views are your current prior. The engagement makes them explicit, encodes them honestly, and identifies where resolving the disagreement would most improve your decisions.
info@rung3.ai
Cooke, R.M., 1991, Experts in Uncertainty, Oxford University Press · O'Hagan, A. et al., 2006, Uncertain Judgments: Eliciting Experts' Probabilities, Wiley · Pearl, J., 2009, Causality (2nd ed.), Cambridge University Press · Clemen, R.T. & Winkler, R.L., 1999, “Combining Probability Distributions from Experts in Risk Analysis,” Risk Analysis 19(2)