mlsanityData to 14 Oct 2026

UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation

arXiv 2405.20612 ↗First posted 31 May 2024cs.CL cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive capabilities in various tasks using the in-context learning (ICL) paradigm. However, their effectiveness is often compromised by inherent bias, leading to prompt brittleness, i.e., sensitivity to design settings such as example selection, order, and prompt formatting. Previous studies have addressed LLM bias through external adjustment of model outputs, but the internal mechanisms that lead to such bias remain unexplored. Our work delves into these mechanisms, particularly investigating how feedforward neural networks (FFNs) and attention heads result in the bias of LLMs. By Interpreting the contribution of individual FFN vectors and attention heads, we identify the biased LLM components that skew LLMs' prediction toward specific labels. To mitigate these biases, we introduce UniBias, an inference-only method that effectively identifies and eliminates biased FFN vectors and attention heads. Extensive experiments across 12 NLP datasets demonstrate that UniBias significantly enhances ICL performance and alleviates prompt brittleness of LLMs.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

24 of 36 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  10. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  11. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
12 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  3. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  4. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  5. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  6. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  7. 2022Analyzing Transformers in Embedding Spacecited by 132
  8. 2023Mitigating Label Biases for In-context Learningcited by 89
  9. 2022Prototypical Calibration for Few-shot Learning of Language Modelscited by 58
  10. 2023Characterizing Mechanisms for Factual Recall in Language Modelscited by 48
  11. 2023LLMs Learn Task Heuristics from Demonstrations: A Heuristic-Driven Prompting Strategy for Document-Level Event Argument Extractioncited by 8
  12. 2024Unveiling and Manipulating Prompt Influence in Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 16; 2026: 12.

31 papers.

Sort:

By area:

  1. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 101
  2. 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 59
  3. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 49
  4. 2025How Many Instructions Can LLMs Follow at Once?cited by 20
  5. 2024NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Modelscited by 9
  6. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  7. 2025Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Predictioncited by 7
  8. 2025Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correctioncited by 7
  9. 2025Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approachcited by 5
  10. 2025Benchmarking and Mitigating MCQA Selection Bias of Large Vision-Language Modelscited by 5
  11. 2025Robustness is Important: Limitations of LLMs for Data Fittingcited by 2
  12. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulationscited by 2
19 more
  1. 2026When Numbers Start Talking: Implicit Numerical Coordination Among LLM-Based Agentscited by 2
  2. 2026Understanding the Dynamics of Demonstration Conflict in In-Context Learningcited by 2
  3. 2025Exploiting Contextual Knowledge in LLMs through V-usable Information based Layer Enhancementcited by 1
  4. 2025In-Context Bias Propagation in LLM-Based Tabular Data Generationcited by 1
  5. 2025BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injectioncited by 1
  6. 2025Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweetscited by 1
  7. 2026Identity-Robust Language Model Generation via Content Integrity Preservationcited by 1
  8. 2026Tracing the Latent Threads: A Mechanistic Study of How LLMs Represent and Operationalize Race and Ethnicity Cuescited by 1
  9. 2026Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimizationcited by 1
  10. 2025Large Language Models Are Still Misled by Simple Bias Ensemblescited by 0
  11. 2025Fair-GPTQ: Bias-Aware Quantization for Large Language Modelscited by 0
  12. 2025Beyond Hidden-Layer Manipulation: Semantically-Aware Logit Interventions for Debiasing LLMscited by 0
  13. 2026CBMAS: Cognitive Behavioral Modeling via Activation Steeringcited by 0
  14. 2026UGID: Unified Graph Isomorphism for Debiasing Large Language Modelscited by 0
  15. 2026Robust LLM Performance Certification via Constrained Maximum Likelihood Estimationcited by 0
  16. 2026FairNVT: Fair Classification via Noise Injection in Vision Transformerscited by 0
  17. 2026Counting Worlds Branching Time Semantics for post-hoc Bias Mitigation in generative AIcited by 0
  18. 2026When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigationcited by 0
  19. 2026Thinking with Images without Coordinatescited by 0