# Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

arXiv: https://arxiv.org/abs/2606.15127 · HTML: https://mlsanity.com/paper/2606.15127/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 13 Jun 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models](https://mlsanity.com/paper/2405.17814/) (2024)
- [BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Models](https://mlsanity.com/paper/2407.15240/) (2024)
- [When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.00448/) (2026)
- [Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict](https://mlsanity.com/paper/2605.14473/) (2026)
- [Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG](https://mlsanity.com/paper/2605.28044/) (2026)
- [CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding](https://mlsanity.com/paper/2512.04111/) (2025)
- [Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection](https://mlsanity.com/paper/2605.24834/) (2026)
- [ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuation](https://mlsanity.com/paper/2605.26542/) (2026)
- [aiXiv: A Next-Generation Open Access Ecosystem for Scientific Discovery Generated by AI Scientists](https://mlsanity.com/paper/2508.15126/) (2025)
- [SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime](https://mlsanity.com/paper/2602.19555/) (2026)
- [AtelierEval: Agentic Evaluation of Humans & LLMs as Text-to-Image Prompters](https://mlsanity.com/paper/2605.22645/) (2026)
- [PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction](https://mlsanity.com/paper/2602.06714/) (2026)
- [When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Models](https://mlsanity.com/paper/2603.23085/) (2026)
- [CURE:Circuit-Aware Unlearning for LLM-based Recommendation](https://mlsanity.com/paper/2604.04982/) (2026)
- [CRAB: Codebook Rebalancing for Bias Mitigation in Generative Recommendation](https://mlsanity.com/paper/2604.05113/) (2026)
- [BiasIG: Benchmarking Multi-dimensional Social Biases in Text-to-Image Models](https://mlsanity.com/paper/2604.11934/) (2026)
- [Test of Time: Rethinking Temporal Signal of Benchmark Contamination](https://mlsanity.com/paper/2509.00072/) (2025)
- [MedEqualizer: A Framework Investigating Bias in Synthetic Medical Data and Mitigation via Augmentation](https://mlsanity.com/paper/2511.01054/) (2025)
- [MRAG-Suite: A Diagnostic Evaluation Platform for Visual Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.24253/) (2025)

## Built on by (most cited first, 9 of 9)

- [Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happened](https://mlsanity.com/paper/2607.13083/) (2026)
- [When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost](https://mlsanity.com/paper/2607.24010/) (2026)
- [AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning](https://mlsanity.com/paper/2608.29988/) (2026)
- [Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventions](https://mlsanity.com/paper/2607.21635/) (2026)
- [Noise Floor Audit for Agent Benchmarks](https://mlsanity.com/paper/2608.22331/) (2026)
- [LOOMSUM: Weaving Quantitative and Narrative Evidence for Faithful Long Text-Table Summarization](https://mlsanity.com/paper/2609.00241/) (2026)
- [Bioinfoysis Technical Report](https://mlsanity.com/paper/2609.03871/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [Can Language Models Learn to Reject Their Own Bad Reasoning Steps?](https://mlsanity.com/paper/2610.05976/) (2026)

## Abstract (arXiv)

Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students intermediate steps, decision-support systems may require human oversight, and audit workflows may inspect traces for misleading or biased input. In such settings, two responses can receive the same final-answer score while differing in whether the trace explicitly flags injected biasing content. Accuracy-only evaluation collapses these cases. We study this gap as a measurement blind spot for responsible evaluation and introduce a minimal trace-level diagnostic with two axes: susceptibility (whether the bias breaks a previously correct answer) and acknowledgment (whether the trace contains a rubric-defined surface reference to the injected content). Across thousands of biased GSM8K trials, GPT-4o and Claude Sonnet~4 have similar susceptibility rates (1.3% vs. 1.2%) but substantially different acknowledgment rates (13.0% vs. 75.0%) under the same rubric.