# Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning

arXiv: https://arxiv.org/abs/2401.00139 · HTML: https://mlsanity.com/paper/2401.00139/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning.
- Builds on 9 of its 85 references that are in the map.
- First posted on arXiv 30 Dec 2023.

## Placed in

- LLM reasoning / [Causal reasoning in LLMs](https://mlsanity.com/lr/causal-reasoning-in-llms/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 9 of 9)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence](https://mlsanity.com/paper/2002.06177/) (2020)
- [Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation](https://mlsanity.com/paper/2305.07375/) (2023)

## Built on by (most cited first, 14 of 14)

- [Causal Inference with Large Language Model: A Survey](https://mlsanity.com/paper/2409.09822/) (2024)
- [ALCM: Autonomous LLM-Augmented Causal Discovery Framework](https://mlsanity.com/paper/2405.01744/) (2024)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [Dual Active Learning for Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2410.02504/) (2024)
- [On the Reliability of Large Language Models for Causal Discovery](https://mlsanity.com/paper/2407.19638/) (2024)
- [CausalGraph2LLM: Evaluating LLMs for Causal Queries](https://mlsanity.com/paper/2410.15939/) (2024)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Evaluating Interventional Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2404.05545/) (2024)
- [Simulating Policy Impacts: Developing a Generative Scenario Writing Method to Evaluate the Perceived Effects of Regulation](https://mlsanity.com/paper/2405.09679/) (2024)
- [Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models](https://mlsanity.com/paper/2510.18526/) (2025)
- [ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models](https://mlsanity.com/paper/2502.15487/) (2025)
- [Causal Distillation: Transferring Structured Explanations from Large to Compact Language Models](https://mlsanity.com/paper/2505.19511/) (2025)
- [InfoCausalQA:Can Models Perform Non-explicit Causal Reasoning Based on Infographic?](https://mlsanity.com/paper/2508.06220/) (2025)
- [CARE: Turning LLMs Into Causal Reasoning Expert](https://mlsanity.com/paper/2511.16016/) (2025)

## Abstract (arXiv)

This paper introduces a causal attribution model to enhance the interpretability of large language models (LLMs) and improve their causal reasoning abilities via precise fine-tuning. Despite LLMs' proficiency in diverse tasks, their reasoning processes often remain black box, and thus restrict targeted enhancement. We propose a novel causal attribution model that utilizes "do-operators" for constructing interventional scenarios, allowing us to quantify the contribution of different components in LLMs's causal reasoning process systematically. By assessing the proposed attribution scores through causal discovery tasks across various domains, we demonstrate that LLMs' effectiveness in causal discovery heavily relies on provided context and domain-specific knowledge but can also utilize numerical data with limited calculations in correlation, not causation. This motivates the proposed fine-tuned LLM for pairwise causal discovery, effectively and correctly leveraging both knowledge and numerical information.