# Prompting Fairness: Integrating Causality to Debias Large Language Models

arXiv: https://arxiv.org/abs/2403.08743 · HTML: https://mlsanity.com/paper/2403.08743/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 80 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Large Language Model Unlearning](https://mlsanity.com/paper/2310.10683/) (2023)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [The Capacity for Moral Self-Correction in Large Language Models](https://mlsanity.com/paper/2302.07459/) (2023)
- [Causal Inference in the Presence of Latent Variables and Selection Bias](https://mlsanity.com/paper/1302.4983/) (2013)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models](https://mlsanity.com/paper/2310.06117/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)

## Built on by (most cited first, 21 of 21)

- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Bias in Large Language Models: Origin, Evaluation, and Mitigation](https://mlsanity.com/paper/2411.10915/) (2024)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions](https://mlsanity.com/paper/2410.02584/) (2024)
- ["You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations](https://mlsanity.com/paper/2406.12232/) (2024)
- [Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs](https://mlsanity.com/paper/2410.12864/) (2024)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs](https://mlsanity.com/paper/2502.01926/) (2025)
- [Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs](https://mlsanity.com/paper/2412.14304/) (2024)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Mitigating Biases in Language Models via Bias Unlearning](https://mlsanity.com/paper/2509.25673/) (2025)
- [Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts](https://mlsanity.com/paper/2602.04398/) (2026)
- [BiasCause: Evaluate Socially Biased Causal Reasoning of Large Language Models](https://mlsanity.com/paper/2504.07997/) (2025)
- [Expert-Guided Extinction of Toxic Tokens for Debiased Generation](https://mlsanity.com/paper/2405.19299/) (2024)
- ['Rich Dad, Poor Lad': How do Large Language Models Contextualize Socioeconomic Factors in College Admission ?](https://mlsanity.com/paper/2509.16400/) (2025)
- [Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models](https://mlsanity.com/paper/2512.04981/) (2025)
- [DeFrame: Debiasing Large Language Models Against Framing Effects](https://mlsanity.com/paper/2602.04306/) (2026)
- [On the Thinking-Language Modeling Gap in Large Language Models](https://mlsanity.com/paper/2505.12896/) (2025)
- [Alleviating Choice Supportive Bias in LLM with Reasoning Dependency Generation](https://mlsanity.com/paper/2512.03082/) (2025)
- [Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting](https://mlsanity.com/paper/2512.19735/) (2025)
- [Causally Fair Generation with Large Language Models](https://mlsanity.com/paper/2610.04444/) (2026)

## Abstract (arXiv)

Large language models (LLMs), despite their remarkable capabilities, are susceptible to generating biased and discriminatory responses. As LLMs increasingly influence high-stakes decision-making (e.g., hiring and healthcare), mitigating these biases becomes critical. In this work, we propose a causality-guided debiasing framework to tackle social biases, aiming to reduce the objectionable dependence between LLMs' decisions and the social information in the input. Our framework introduces a novel perspective to identify how social information can affect an LLM's decision through different causal pathways. Leveraging these causal insights, we outline principled prompting strategies that regulate these pathways through selection mechanisms. This framework not only unifies existing prompting-based debiasing techniques, but also opens up new directions for reducing bias by encouraging the model to prioritize fact-based reasoning over reliance on biased social cues. We validate our framework through extensive experiments on real-world datasets across multiple domains, demonstrating its effectiveness in debiasing LLM decisions, even with only black-box access to the model.