Robustness
For Both Executive and Technical ReadersA posterior is a point. The question a regulated decision turns on is how fragile that point is: to a wrong expert estimate inside the model, and to a wrong or missing observation feeding it.
01 The Gap
The model answered the question. It didn’t say how much you should trust it.
Most probabilistic systems answer the question they were asked and stop. They do not tell you whether their answer would change if an input were slightly wrong. In a regulated context, that omission is the vulnerability: a conclusion built on a single shaky input that nobody tested.
A conclusion can be wrong two ways. The parameters can be off: an expert’s probability estimate inside the model is slightly wrong. The evidence can be off: an observation feeding the model is missing, mistaken, or about to be revised. Robustness analysis tests both.
02 The component
Two analyses, sensitivity to assumptions, and necessity of evidence.
Robustness is a Cognitive Primitives, Epistemics component. It runs two complementary analyses on any inference result: retracted analysis, asking what the model would conclude if a piece of evidence turned out to be wrong or absent, (what if this observation were wrong or absent?) and sensitivity analysis (what if this parameter estimate were off?). Together they produce a RobustnessReport: a typed artifact showing which conclusions are solid and which rest on a single shaky input.
| Analysis | What it answers |
|---|---|
| Retracted evidence | For each piece of evidence, what the posterior would be without it, and how far it moves, so you can see which observations the conclusion actually leans on |
| Sensitivity | For each parameter, the range the posterior spans if that number were off, and whether that range is wide enough to flip the conclusion |
The verdict field is machine-readable: a downstream governance artifact can require "robust" before a decision proceeds without human review.
03 How it works
Perturb the CPTs. Re-run. Measure what changed.
Two analyses, run on the current inference result:
Retracted analysis. Remove each piece of evidence in turn and re-run inference. A positive blood culture pushed sepsis posterior to 0.93, remove it, and the posterior falls to 0.66. The conclusion leans heavily on that one result. Remove the temperature reading, and almost nothing moves. Now the clinician knows exactly which inputs the diagnosis depends on.
Sensitivity analysis. Perturb each expert parameter estimate across a plausible range and observe the posterior range. If P(Shock | Sepsis) = 0.73 could be anywhere from 0.60 to 0.85, and the conclusion barely moves, the estimate can be rough. If it flips the decision, that is the parameter most deserving an expert’s care before production.
Verdict. The RobustnessReport combines both into a single verdict, auditable by a regulator: which inputs the conclusion depends on, and how much it would take to change it.
04 The Brains
The model checking its own work.
Robustness is the brains checking its own work, testing which of its conclusions would survive challenge and which would not.
There is no .bayes file for Robustness, Robustness operates on any domain model. The artifact it produces is a RobustnessReport attached to the AuditRecord. In regulated contexts, the report is a required output alongside the posterior.
| Analysis | Result |
|---|---|
| Retracted evidence | Removing BloodCulture: posterior drops to 0.66, a delta of −0.27, the conclusion leans heavily on this observation |
| Sensitivity | P(Shock | Sepsis): posterior ranges 0.81–0.85 across plausible values, not critical, this parameter can be rough |
05 Query in plain English
How much should I trust this answer?
Robustness answers the question that comes after the posterior: how much should I trust it?
The audit trail is the RobustnessReport, not the analyst’s judgment about which inputs matter.
06 Where it sits
Downstream of Execute, upstream of the governance handoff.
Robustness sits downstream of Execute (where the posterior is computed) and upstream of the governance handoff (12 BPMN Integration), which may require a "robust" verdict before the pipeline proceeds without human review.
Upstream: the inference result and AuditRecord from Execute (01 Pipeline stage 6). Downstream: the RobustnessReport is appended to the AuditRecord and consumed by 12 BPMN Integration. It also informs 07 Validation (calibration checks) and feeds back to 04 Cognitive Primitives (which parameters most need re-elicitation).