What Do Rationales Communicate? A Message-Intervention Study in Role-Specialized QA
Not on the map. Why?
Abstract (arXiv)
Role-specialized QA pipelines increasingly pass rationales from a reasoner to a verifier, but it is unclear what this message actually buys: better answers, stronger support assessment, or a new failure surface. We introduce a message-intervention diagnostic that fixes the evidence and candidate answer while varying only the rationale passed across the reasoner-to-verifier boundary. On 400 MuSiQue, HotpotQA, and 2WikiMultiHopQA examples with DeepSeek as generator and verifier, faithful rationales add almost no answer accuracy over no rationale, while corrupted rationales strongly alter support judgments. Under a blind verifier prompt, harmless paraphrases shift support by only 0--2.5%, whereas corrupted rationales shift support by 10--22%; an explicit rationale-checking prompt amplifies the same pattern to 34--55%. Final answers move less (2--30%), and only 2.9--35.3% of corrupted support flips co-occur with answer changes. Human audits show why this matters: 16/42 valid corruptions are corruption-overtrust cases, and blind humans reject or mark unclear 9/10 audited corrupted rationales that the model accepts. Cross-model and task-boundary checks show when the channel is active, amplified, inert, or folded into the task label. Rationale sharing should be evaluated as a verification-message mechanism, not merely as a route to higher answer accuracy.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
21 of 30 references.
References · 21
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Let's Verify Step by Stepcited by 3,789
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
- 2023Faithfulness Tests for Natural Language Explanationscited by 107
9 more
- 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 73
- 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 66
- 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 47
- 2025Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analysescited by 14
- 2026Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditingcited by 7
- 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debatecited by 7
- 2025CLEV: LLM-Based Evaluation Through Lightweight Efficient Voting for Free-Form Question-Answeringcited by 6
- 2026Evaluating Chain-of-Thought Reasoning through Reusability and Verifiabilitycited by 4
- 2026Preventing Error Propagation in Multi-Agent AI through Runtime Monitoringcited by 1
Built on by
Not cited yet.