# Large Causal Models from Large Language Models

arXiv: https://arxiv.org/abs/2512.07796 · HTML: https://mlsanity.com/paper/2512.07796/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Builds on (most cited first, 10 of 10)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics](https://mlsanity.com/paper/1908.07021/) (2019)
- [Backprop as Functor: A compositional perspective on supervised learning](https://mlsanity.com/paper/1711.10455/) (2017)
- [Position: Categorical Deep Learning is an Algebraic Theory of All Architectures](https://mlsanity.com/paper/2402.15332/) (2024)
- [Multi-Agent Causal Discovery Using Large Language Models](https://mlsanity.com/paper/2407.15073/) (2024)
- [A Survey on Extraction of Causal Relations from Natural Language Text](https://mlsanity.com/paper/2101.06426/) (2021)
- [Intuitionistic j-Do-Calculus in Topos Causal Models](https://mlsanity.com/paper/2510.17944/) (2025)
- [GAIA: Categorical Foundations of Generative AI](https://mlsanity.com/paper/2402.18732/) (2024)
- [Decentralized Causal Discovery using Judo Calculus](https://mlsanity.com/paper/2510.23942/) (2025)

## Built on by (most cited first, 8 of 8)

- [Latent Confounded Causal Discovery via Lie Bracket Geometry](https://mlsanity.com/paper/2606.19610/) (2026)
- [PROMETHEUS: Automating Deep Causal Research Integrating Text, Data and Models](https://mlsanity.com/paper/2605.12835/) (2026)
- [Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning](https://mlsanity.com/paper/2605.27259/) (2026)
- [Causality Elicitation from Large Language Models](https://mlsanity.com/paper/2603.04276/) (2026)
- [Odyssey: Constructing Verifiable Local Truth-Preserving Foundation Models](https://mlsanity.com/paper/2606.27593/) (2026)
- [HOLOGRAPH: Active Causal Discovery via Sheaf-Theoretic Alignment of Large Language Model Priors](https://mlsanity.com/paper/2512.24478/) (2025)
- [CSQL: Mapping Documents into Causal Databases](https://mlsanity.com/paper/2601.08109/) (2026)
- [FinInvest-GTCN: Explainable Graph-Temporal-Causal Modeling for Risk-Aware Investment Decision Optimization](https://mlsanity.com/paper/2606.28933/) (2026)

## Abstract (arXiv)

We introduce a new paradigm for building large causal models (LCMs) that exploits the enormous potential latent in today's large language models (LLMs). We describe our ongoing experiments with an implemented system called DEMOCRITUS (Decentralized Extraction of Manifold Ontologies of Causal Relations Integrating Topos Universal Slices) aimed at building, organizing, and visualizing LCMs that span disparate domains extracted from carefully targeted textual queries to LLMs. DEMOCRITUS is methodologically distinct from traditional narrow domain and hypothesis centered causal inference that builds causal models from experiments that produce numerical data. A high-quality LLM is used to propose topics, generate causal questions, and extract plausible causal statements from a diverse range of domains. The technical challenge is then to take these isolated, fragmented, potentially ambiguous and possibly conflicting causal claims, and weave them into a coherent whole, converting them into relational causal triples and embedding them into a LCM. Addressing this technical challenge required inventing new categorical machine learning methods, which we can only briefly summarize in this paper, as it is focused more on the systems side of building DEMOCRITUS. We describe the implementation pipeline for DEMOCRITUS comprising of six modules, examine its computational cost profile to determine where the current bottlenecks in scaling the system to larger models. We describe the results of using DEMOCRITUS over a wide range of domains, spanning archaeology, biology, climate change, economics, medicine and technology. We discuss the limitations of the current DEMOCRITUS system, and outline directions for extending its capabilities.