# Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort

arXiv: https://arxiv.org/abs/2510.01367 · HTML: https://mlsanity.com/paper/2510.01367/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [Defining and Characterizing Reward Hacking](https://mlsanity.com/paper/2209.13085/) (2022)
- [Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models](https://mlsanity.com/paper/2406.10162/) (2024)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [On Memorization of Large Language Models in Logical Reasoning](https://mlsanity.com/paper/2410.23123/) (2024)
- [Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models](https://mlsanity.com/paper/2502.17387/) (2025)
- [Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs](https://mlsanity.com/paper/2402.14740/) (2024)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Preventing Language Models From Hiding Their Reasoning](https://mlsanity.com/paper/2310.18512/) (2023)
- [Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.22777/) (2025)

## Built on by (most cited first, 27 of 27)

- [Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought](https://mlsanity.com/paper/2603.05488/) (2026)
- [LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking](https://mlsanity.com/paper/2604.15149/) (2026)
- [Reward Hacking in Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2605.12474/) (2026)
- [Counterfactual Simulation Training for Chain-of-Thought Faithfulness](https://mlsanity.com/paper/2602.20710/) (2026)
- [Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR](https://mlsanity.com/paper/2603.07084/) (2026)
- [Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2606.04923/) (2026)
- [Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory](https://mlsanity.com/paper/2602.18297/) (2026)
- [Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale](https://mlsanity.com/paper/2605.20744/) (2026)
- [Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners](https://mlsanity.com/paper/2601.02996/) (2026)
- [CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies](https://mlsanity.com/paper/2606.16613/) (2026)
- [Learning Next Action Predictors from Human-Computer Interaction](https://mlsanity.com/paper/2603.05923/) (2026)
- [From Rebound to Remedy: Understanding and Mitigating Reward Hacking via Representation Engineering](https://mlsanity.com/paper/2604.01476/) (2026)
- [Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning](https://mlsanity.com/paper/2604.22074/) (2026)
- [Detecting and Suppressing Reward Hacking with Gradient Fingerprints](https://mlsanity.com/paper/2604.16242/) (2026)
- [Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests](https://mlsanity.com/paper/2606.07379/) (2026)
- [Self-Commitment Latency: A Reward-Free Probe for Prompted Implicit Hacking](https://mlsanity.com/paper/2606.05625/) (2026)
- [Context-Masked Truncated Reasoning Audits for Answer-Key Dependence in LLM Tutors](https://mlsanity.com/paper/2607.04572/) (2026)
- [Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?](https://mlsanity.com/paper/2608.04928/) (2026)
- [INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignment](https://mlsanity.com/paper/2608.27348/) (2026)
- [Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning](https://mlsanity.com/paper/2609.00351/) (2026)
- [RLVR2: Reinforcement Learning with Verifiable Rubric-based Ranking](https://mlsanity.com/paper/2609.23457/) (2026)
- [TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment](https://mlsanity.com/paper/2609.33426/) (2026)
- [When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment](https://mlsanity.com/paper/2609.35074/) (2026)
- [Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control](https://mlsanity.com/paper/2609.35677/) (2026)
- [CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL](https://mlsanity.com/paper/2609.39533/) (2026)
- [hacktrace: behavior-supervised detection of reward hacking during code generation](https://mlsanity.com/paper/2610.03055/) (2026)
- [Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems](https://mlsanity.com/paper/2610.12822/) (2026)

## Abstract (arXiv)

Reward hacking, where a reasoning model exploits loopholes in a reward function to achieve high rewards without solving the intended task, poses a significant threat. This behavior may be explicit, i.e. verbalized in the model's chain-of-thought (CoT), or implicit, where the CoT appears benign thus bypasses CoT monitors. To detect implicit reward hacking, we propose TRACE (Truncated Reasoning AUC Evaluation). Our key observation is that hacking occurs when exploiting the loophole is easier than solving the actual task. This means that the model is using less 'effort' than required to achieve high reward. TRACE quantifies effort by measuring how early a model's reasoning becomes sufficient to obtain the reward. We progressively truncate a model's CoT at various lengths, force the model to answer, and estimate the expected reward at each cutoff. A hacking model, which takes a shortcut, will achieve a high expected reward with only a small fraction of its CoT, yielding a large area under the accuracy-vs-length curve. TRACE achieves over 65% gains over our strongest 72B CoT monitor in math reasoning, and over 30% gains over a 32B monitor in coding. We further show that TRACE can discover unknown loopholes during training. Overall, TRACE offers a scalable unsupervised approach for oversight where current monitoring methods prove ineffective.