The central problem at the intersection of AI, cognitive science, and epistemology is not building larger models. It is transforming what experts already know into structure that machines can manipulate. Causal formalization, not statistical scaling, is what produces AI that reasons about your specific business.
What it means
Formalizing expert reasoning means converting how experts think into explicit computational structures that machines can interpret, manipulate, and reason over. The raw material is tacit: intuition, heuristics, causal understanding, decision logic. The output is explicit: symbolic rules, graphs, probabilistic models, ontologies, or structural causal models.
An example. A clinician reasons: “If the patient has chest pain, elevated troponin, and ECG abnormalities, myocardial infarction becomes likely unless another explanation accounts for the biomarkers.” Inside that sentence are causal assumptions, uncertainty bounds, diagnostic prioritisation, hidden dependencies, and intervention implications. Humans handle this intuitively. Machines need it written down.
Writing it down is the work. The result is a model your domain experts can read, a regulator can audit, and a language model can query — without the language model being the one doing the reasoning.
Five methods, one tradition
The literature names five families of method for formalizing expert reasoning. They are not competitors. They are a lineage — each generation answering the limits of the previous one.
IF–THEN logic. If fever AND cough AND low oxygen, then pneumonia likelihood = high. The classical expert systems of the 1980s. Brittle, poor at uncertainty, weak generalisation — but the first methodical attempt to externalise expert decision-making.
DAGs, Bayesian networks, structural causal models. Pearl’s framework formalizes intervention reasoning and counterfactuals. Encodes mechanisms, not just patterns. Answers what happens if we act and what would have happened if we’d acted differently — questions the rule-based generation could not.
Entities, relations, hierarchies. Diabetes → associated_with → InsulinResistance → causes → Hyperglycemia. Used heavily in biomedical AI and semantic-web systems. Captures structure, but typically without the probability and intervention machinery that causal models bring.
Bayesian reasoning under uncertainty. P(Cancer | Smoking, Age, CTScan). Allows evidence updating, uncertainty propagation, probabilistic diagnosis. The numerical companion to the causal-graph generation; usually combined with it.
LLMs combined with symbolic and causal structures. The frontier. Language models supply linguistic fluency and pattern recognition. Symbolic structures supply mechanism, audit, and counterfactual reasoning. Together they answer the questions neither component answers alone. Worked example →
The engagement operates on rungs 2 and 5: building causal models, then composing them with language-model interfaces. The earlier methods are not obsolete; they are absorbed. A rule extracted from an expert in 2025 is still a rule, but it lives inside a graph that knows what it is conditioned on and what it claims about cause.
Expert reasoning ↔ SCM
The structural causal model is powerful as a formalization target because the moves a domain expert already makes — “this influences that”, “this mediates the effect”, “what would happen if” — have direct, formal counterparts. The translation is concrete:
| What the expert says | What the SCM encodes |
|---|---|
| “This influences that.” | A directed edge in the graph. |
| “This mediates the effect.” | An intermediate node on a directed path. |
| “This is a confounder.” | A common parent of two variables. |
| “Changing X changes Y.” | An intervention: do(X). |
| “What if it had been different?” | A counterfactual query against the abducted noise. |
| “Given this evidence, what’s the most probable cause?” | Backward inference through the graph. |
None of these are inferred from data. They are claims the expert is willing to defend, written down in a form a machine can compute on. The data fills in the magnitudes; the structure is the contribution.
The tacit-knowledge problem
The hardest part of formalization is not writing down what experts say. It is recovering what they do not say. Experts often cannot fully explain their own reasoning. Much expertise is subconscious, experiential, pattern-based — the kind of knowledge a senior underwriter has after twenty years that lets her glance at a claim and know.
The research literature calls this knowledge elicitation, and it has its own set of methods:
- Structured interviews — experts explain decisions they’ve made on specific cases.
- Think-aloud protocols — experts verbalise their reasoning as they work through a live case.
- Concept mapping — variables and relations extracted into a visible graph the expert can correct.
- Causal diagrams — experts draw mechanisms directly, with their inconsistencies surfaced as contradictions.
- Case analysis — reasoning inferred from a portfolio of past decisions, then validated with the expert.
- LLM-assisted extraction — language models pull candidate causal structure from documents and transcripts, with the expert as the final adjudicator.
An engagement does not pick one of these methods. It uses several in sequence, because each surfaces a different layer of what the expert knows. The deliverable is a model the expert reads back and recognizes as her reasoning, captured.
The neurosymbolic frontier
The research question now is whether language models can themselves contribute to formalization — extracting causal claims from technical papers, generating candidate DAGs from interview transcripts, surfacing latent assumptions in scientific literature.
The answer so far is: yes, partially, with supervision. An LLM is useful as an extraction tool because it has read the same papers your experts have. It is not useful as the final reasoner, for the same reason a paralegal is not the judge: pattern matching across precedent is not the same as ruling on this case.
Where this matters for your AI strategy: a language model paired with a causal model is fundamentally different from a language model alone. The LLM brings linguistic fluency and access to general priors. The causal model brings grounded mechanism specific to your organization. Together they answer the questions that drive your highest-stakes decisions. The architecture is real, and the work of building one organization’s causal model is what an engagement actually does. See it worked through →
A growing position in the field holds that causal formalization, not just statistical scaling, is what robust general intelligence will require. The site’s thesis is the consulting version of that academic position: your operation needs the same thing, and the work of building it can begin now.
References
- Pearl, J. (2009). Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press. bayes.cs.ucla.edu/BOOK-2K
- Pearl, J., & Mackenzie, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books.
- Ford, D. N., & Sterman, J. D. (1998). Expert Knowledge Elicitation to Improve Formal and Mental Models. System Dynamics Review, 14(4), 309–340. MIT DSpace
- Narayanan, A., & Jin, Y. (1991). Object-Oriented Representations, Causal Reasoning and Expert Systems. Expert Systems, 8(4). Wiley
- Peralta, A., Olivas, J. A., Romero, F. P., et al. (2025). Integration of Fuzzy Techniques and Formal Representation of Domain and Expert Knowledge in AI Systems. PDF
- Cox, L. A. (2026). Combining Diverse Expert Opinions in Risk Analysis Using Relative Causal Knowledge. Risk Analysis.