The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Chain-of-thought (CoT) monitoring has been proposed as a promising safety mechanism for detecting misaligned behavior in large language models. However, its reliability remains largely unexplored beyond English and across diverse model families. We present the first large-scale evaluation of CoT monitorability across 13 diverse languages and seven frontier model families, comprising 16 models. Using adversarial-hint evaluations that require explicit intermediate computation, together with analysis of internal answer-token probabilities, we consistently find CoT unfaithfulness across languages and hint types, with an average rate of 95.9% across 8B--120B parameter models. We find that frontier models systematically exhibit strategic manipulation, including answer-switching, post-hoc rationalization, and procedural exploitation of hints, making their reasoning difficult to reliably monitor. These deceptive patterns remain especially pronounced in low-resource languages, revealing fundamental limitations in current CoT-based oversight. Our results show that CoT monitoring is fragile under linguistic distribution shift, providing a substantially weaker safety signal than English-only studies suggest. These findings motivate the development of more robust CoT monitors and complementary white-box monitoring techniques, particularly for mid- and low-resource languages. Our code is available hrefhttps://multilingual-cot-monitoring.github.io/textcolorbluehere.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 38 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
34 of 38 references.
Closest ancestors · 8
Same direction, newest first
- 2025Monitoring Monitorability20 Dec
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
- 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors7 Jul
- 2025Reasoning Models Don't Always Say What They Think8 May
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
- 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters20 Dec
Other references · 26
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2025OpenAI GPT-5 System Cardcited by 910
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
- 2025Kimi K2: Open Agentic Intelligencecited by 523
- 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
14 more
- 2023Do Multilingual Language Models Think Better in English?cited by 87
- 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69
- 2025An Approach to Technical AGI Safety and Securitycited by 42
- 2023GlotLID: Language Identification for Low-Resource Languagescited by 37
- 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
- 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
- 2025Evaluating Frontier Models for Stealth and Situational Awarenesscited by 30
- 2025MPO: Multilingual Safety Alignment via Reward Gap Optimizationcited by 18
- 2025When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracycited by 13
- 2025Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignmentcited by 10
- 2026CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoningcited by 8
- 2025CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcarecited by 7
- 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languagescited by 6
- 2025Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking?cited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026On the Chain-of-Thought Monitorability of Looped Language Models
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning
- 2026Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators
- 2026Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning
- 2026Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
- 2026MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?