mlsanityData to 14 Oct 2026

One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs

arXiv 2609.14860 ↗First posted 14 Sep 2026cs.CL cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Warning: This submission studies stereotypes and biases, and contains toxic and offensive examples, used for illustration purposes only. Fairness benchmarks such as BBQ have become the de facto standard for fairness evaluation across major model families. We argue that these benchmarks are too easy to support their role: training Qwen 2.5 7B Base with Group Relative Policy Optimization (GRPO) on a single BBQ example, or placing that example in context as a one-shot demonstration for in-context learning (ICL), lifts mean BBQ accuracy from 79.9% to 92.9% and 99.0%, respectively, closing 80% of the gap to its large-scale RLHF counterpart (96.1%) with GRPO, and surpassing it with ICL. These effects generalize across model families. A cross-conditioning analysis shows the improvement is carried by the reasoning traces generated by the model, and one example suffices to elicit a category-agnostic ``missing evidence'' reasoning pattern. We argue that BBQ-style multiple-choice abstention benchmarks measure a single structural cue, and a model that solves them does not thereby become fair. We call for evaluation suites that cover a broader spectrum of fairness alignment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 42 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

28 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Guiding LLM Decision-Making with Fairness Reward Models15 Jul

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  12. 2023LIMA: Less Is More for Alignmentcited by 1,100
15 more
  1. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  2. 2023Large Language Modelscited by 780
  3. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  4. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  5. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  6. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
  7. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  9. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  10. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  11. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 152
  12. 2025What Makes a Reward Model a Good Teacher? An Optimization Perspectivecited by 73
  13. 2022Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrationscited by 70
  14. 2024Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparisoncited by 21
  15. 2024Evaluating Gender Bias of LLMs in Making Morality Judgementscited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignment
  2. 2025BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models
  3. 2025Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs
  4. 2025Silenced Biases: The Dark Side LLMs Learned to Refuse
  5. 2025Alleviating Choice Supportive Bias in LLM with Reasoning Dependency Generation
  6. 2025Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning
  7. 2024Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classification
  8. 2025Towards Large Language Models that Benefit for All: Benchmarking Group Fairness in Reward Models