mlsanityData to 14 Oct 2026

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

arXiv 2606.15127 ↗First posted 13 Jun 2026cs.LGcited by 9

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students intermediate steps, decision-support systems may require human oversight, and audit workflows may inspect traces for misleading or biased input. In such settings, two responses can receive the same final-answer score while differing in whether the trace explicitly flags injected biasing content. Accuracy-only evaluation collapses these cases. We study this gap as a measurement blind spot for responsible evaluation and introduce a minimal trace-level diagnostic with two axes: susceptibility (whether the bias breaks a previously correct answer) and acknowledgment (whether the trace contains a rubric-defined surface reference to the injected content). Across thousands of biased GSM8K trials, GPT-4o and Claude Sonnet~4 have similar susceptibility rates (1.3% vs. 1.2%) but substantially different acknowledgment rates (13.0% vs. 75.0%) under the same rubric.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 44 references that are in the map. First posted on arXiv 13 Jun 2026.

Builds on

23 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Reasoning Models Don't Always Say What They Think8 May
  2. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul

Other references · 20

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Modelscited by 22
  3. 2024BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Modelscited by 18
  4. 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 17
  5. 2026Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflictcited by 15
  6. 2026Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAGcited by 14
  7. 2025CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Codingcited by 12
  8. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflectioncited by 12
  9. 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuationcited by 11
  10. 2025aiXiv: A Next-Generation Open Access Ecosystem for Scientific Discovery Generated by AI Scientistscited by 9
  11. 2026SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtimecited by 8
  12. 2026AtelierEval: Agentic Evaluation of Humans & LLMs as Text-to-Image Prompterscited by 8
8 more
  1. 2026PrefIx: Understand and Adapt to User Preference in Human-Agent Interactioncited by 7
  2. 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Modelscited by 6
  3. 2026CURE:Circuit-Aware Unlearning for LLM-based Recommendationcited by 6
  4. 2026CRAB: Codebook Rebalancing for Bias Mitigation in Generative Recommendationcited by 5
  5. 2026BiasIG: Benchmarking Multi-dimensional Social Biases in Text-to-Image Modelscited by 5
  6. 2025Test of Time: Rethinking Temporal Signal of Benchmark Contaminationcited by 3
  7. 2025MedEqualizer: A Framework Investigating Bias in Synthetic Medical Data and Mitigation via Augmentationcited by 3
  8. 2025MRAG-Suite: A Diagnostic Evaluation Platform for Visual Retrieval-Augmented Generationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happenedcited by 7
  2. 2026When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Costcited by 2
  3. 2026AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoningcited by 2
  4. 2026Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventionscited by 1
  5. 2026Noise Floor Audit for Agent Benchmarkscited by 0
  6. 2026LOOMSUM: Weaving Quantitative and Narrative Evidence for Faithful Long Text-Table Summarizationcited by 0
  7. 2026Bioinfoysis Technical Reportcited by 0
  8. 2026Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agentscited by 0
  9. 2026Can Language Models Learn to Reject Their Own Bad Reasoning Steps?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?
  2. 2026Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations
  3. 2026The Answer Is Not the Argument
  4. 2026BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
  5. 2026No Judgment Without a Reason: Counterfactual Receipts for Versioned AI Evaluators
  6. 2026An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
  7. 2026A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models
  8. 2026Reasoning-Token Spikes Under Prompted Untruthful Responding in Large Language Models