# Causal-Guided Active Learning for Debiasing Large Language Models

arXiv: https://arxiv.org/abs/2408.12942 · HTML: https://mlsanity.com/paper/2408.12942/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [The Capacity for Moral Self-Correction in Large Language Models](https://mlsanity.com/paper/2302.07459/) (2023)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [Learning from others' mistakes: Avoiding dataset biases without modeling them](https://mlsanity.com/paper/2012.01300/) (2020)
- [TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models](https://mlsanity.com/paper/2306.11507/) (2023)
- [MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning](https://mlsanity.com/paper/2311.02303/) (2023)
- [In-Contextual Gender Bias Suppression for Large Language Models](https://mlsanity.com/paper/2309.07251/) (2023)

## Built on by (most cited first, 15 of 15)

- [FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering](https://mlsanity.com/paper/2504.14492/) (2025)
- [From Selection to Generation: A Survey of LLM-based Active Learning](https://mlsanity.com/paper/2502.11767/) (2025)
- [Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage](https://mlsanity.com/paper/2601.01685/) (2026)
- [BiasFilter: An Inference-Time Debiasing Framework for Large Language Models](https://mlsanity.com/paper/2505.23829/) (2025)
- [C2PO: Diagnosing and Disentangling Bias Shortcuts in LLMs](https://mlsanity.com/paper/2512.23430/) (2025)
- [AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models](https://mlsanity.com/paper/2507.21773/) (2025)
- [Self-Debias: Self-correcting for Debiasing Large Language Models](https://mlsanity.com/paper/2604.08243/) (2026)
- [Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents](https://mlsanity.com/paper/2605.20061/) (2026)
- [CRAT: A Multi-Agent Framework for Causality-Enhanced Reflective and Retrieval-Augmented Translation with Large Language Models](https://mlsanity.com/paper/2410.21067/) (2024)
- [Causal MAS: A Survey of Large Language Model Architectures for Discovery and Effect Estimation](https://mlsanity.com/paper/2509.00987/) (2025)
- [ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models](https://mlsanity.com/paper/2510.13103/) (2025)
- [Learn More with Less: Uncertainty Consistency Guided Query Selection for RLVR](https://mlsanity.com/paper/2601.22595/) (2026)
- [Beyond Perception Errors: Semantic Fixation in Large Vision-Language Models](https://mlsanity.com/paper/2604.12119/) (2026)
- [Information Gain-Guided Causal Intervention for Autonomous Debiasing Large Language Models](https://mlsanity.com/paper/2504.12898/) (2025)
- [Debias It Yourself: Teaching LLMs Cognitive Bias Mitigation Interventions](https://mlsanity.com/paper/2609.40124/) (2026)

## Abstract (arXiv)

Although achieving promising performance, recent analyses show that current generative large language models (LLMs) may still capture dataset biases and utilize them for generation, leading to poor generalizability and harmfulness of LLMs. However, due to the diversity of dataset biases and the over-optimization problem, previous prior-knowledge-based debiasing methods and fine-tuning-based debiasing methods may not be suitable for current LLMs. To address this issue, we explore combining active learning with the causal mechanisms and propose a casual-guided active learning (CAL) framework, which utilizes LLMs itself to automatically and autonomously identify informative biased samples and induce the bias patterns. Then a cost-effective and efficient in-context learning based method is employed to prevent LLMs from utilizing dataset biases during generation. Experimental results show that CAL can effectively recognize typical biased instances and induce various bias patterns for debiasing LLMs.