Can Instruction Fine-Tuned Language Models Identify Social Bias through Prompting?
Not on the map. Why?
Abstract (arXiv)
As the breadth and depth of language model applications continue to expand rapidly, it is increasingly important to build efficient frameworks for measuring and mitigating the learned or inherited social biases of these models. In this paper, we present our work on evaluating instruction fine-tuned language models' ability to identify bias through zero-shot prompting, including Chain-of-Thought (CoT) prompts. Across LLaMA and its two instruction fine-tuned versions, Alpaca 7B performs best on the bias identification task with an accuracy of 56.7%. We also demonstrate that scaling up LLM size and data diversity could lead to further performance gain. This is a work-in-progress presenting the first component of our bias mitigation framework. We will keep updating this work as we get more results.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 19 Jul 2023.
Builds on
17 of 26 references.
References · 17
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
5 more
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
- 2021Pre-Trained Models: Past, Present and Futurecited by 319
- 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
- 2020A Survey on Contextual Embeddingscited by 48
Built on by
Citing papers per year: 2023: 1; 2024: 2; 2025: 3.
6 papers.
- 2024Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Frameworkcited by 8
- 2025Detecting Linguistic Indicators for Stereotype Assessment with Large Language Modelscited by 5
- 2023On The Role of Reasoning in the Identification of Subtle Stereotypes in Natural Languagecited by 3
- 2024Red-Teaming for Inducing Societal Bias in Large Language Modelscited by 1
- 2025DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Modelscited by 0
- 2025Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluationcited by 0
- 2025Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation11 Dec 2025
- 2025DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Models25 Mar 2025
- 2025Detecting Linguistic Indicators for Stereotype Assessment with Large Language Models26 Feb 2025
- 2024Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Framework20 Dec 2024
- 2024Red-Teaming for Inducing Societal Bias in Large Language Models8 May 2024
- 2023On The Role of Reasoning in the Identification of Subtle Stereotypes in Natural Language24 Jul 2023
Similar, not cited
Close in wording, no citation either way
- 2025Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection
- 2026Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs
- 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
- 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
- 2026Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents
- 2025Bias after Prompting: Persistent Discrimination in Large Language Models
- 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
- 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models