BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) may encode biased associations from heterogeneous training corpora that are not immediately visible under ordinary prompting, but can surface when the model is steered toward particular demographic personas. Such behavior often manifests not as explicit toxic output, but as systematic performance differences across semantically equivalent tasks, making the resulting bias difficult to detect and mitigate. To address this issue, we formalize the implicit bias problem as persona-induced performance disparity and argue that bias evidence should be treated as a graded signal rather than a binary label. Motivated by this observation, we model biased knowledge as a fuzzy subset equipped with an explicit membership function that reflects the strength of bias evidence for each candidate example. Building on this formulation, we propose Bayesian-Theory-based Bias Removal (BTBR), a hybrid probabilistic-fuzzy framework for identifying and removing latent bias traces from model parameters. BTBR first performs likelihood-ratio screening to measure how strongly candidate samples align with a target biased persona, then converts high-membership samples into structured knowledge triples, and finally applies targeted model editing with a lightweight fuzzy rule scheduler to reduce collateral performance degradation under high entanglement risk. Extensive experiments across multiple bias sources, tasks, model families and editing backends show that BTBR consistently reduces persona-induced performance gaps while preserving general reasoning ability. These results demonstrate that combining probabilistic evidence with fuzzy degree modeling provides an effective and practical approach for mitigating implicit bias in large language models.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 50 of its 64 references that are in the map. First posted on arXiv 20 Aug 2024.
Builds on
50 of 64 references.
References · 50
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
- 2022Mass-Editing Memory in a Transformercited by 976
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
38 more
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2018Gender Bias in Coreference Resolutioncited by 438
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
- 2023Towards Unbounded Machine Unlearningcited by 261
- 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
- 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
- 2021Machine Unlearning of Features and Labelscited by 223
- 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
- 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
- 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
- 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
- 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
- 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2019Wasserstein Fair Classificationcited by 131
- 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 92
- 2023A Survey on Fairness in Large Language Modelscited by 91
- 2024What makes unlearning hard and what to do about itcited by 89
- 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
- 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
- 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
- 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
- 2020PowerTransformer: Unsupervised Controllable Revision for Biased Language Correctioncited by 59
- 2024A Unified Framework for Model Editingcited by 54
- 2022Perturbation Augmentation for Fairer NLPcited by 51
- 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 50
- 2024Rebuilding ROME : Resolving Model Collapse during Sequential Model Editingcited by 50
- 2020Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Taskscited by 47
- 2021NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternativescited by 46
- 2024Localizing Paragraph Memorization in Language Modelscited by 30
- 2024PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learningcited by 28
- 2024Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Taskscited by 28
- 2024BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Modelscited by 27
- 2023An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Modelscited by 16
- 2024Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3cited by 8
Built on by
Citing papers per year: 2024: 2; 2025: 1; 2026: 1.
4 papers.
- 2024CogniDual Framework: Self-Training Large Language Models within a Dual-System Theoretical Framework for Improving Cognitive Taskscited by 5
- 2024Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Frameworkcited by 4
- 2025DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMscited by 2
- 2026Beyond In-Domain Detection: SpikeScore for Cross-Domain Hallucination Detectioncited by 1
- 2026Beyond In-Domain Detection: SpikeScore for Cross-Domain Hallucination Detection27 Jan 2026
- 2025DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs15 May 2025
- 2024Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Framework7 Sep 2024
- 2024CogniDual Framework: Self-Training Large Language Models within a Dual-System Theoretical Framework for Improving Cognitive Tasks5 Sep 2024
Similar, not cited
Close in wording, no citation either way
- 2025BiasFilter: An Inference-Time Debiasing Framework for Large Language Models
- 2025Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement
- 2024BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization
- 2026Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation
- 2026Biases in the Blind Spot: Detecting What LLMs Fail to Mention
- 2026Multi-Persona Thinking for Bias Mitigation in Large Language Models
- 2024Evaluating Nuanced Bias in Large Language Model Free Response Answers
- 2026ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues