# Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectories

arXiv: https://arxiv.org/abs/2604.17596 · HTML: https://mlsanity.com/paper/2604.17596/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023)
- [Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models](https://mlsanity.com/paper/2406.10162/) (2024)
- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025)
- [Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem](https://mlsanity.com/paper/2512.24873/) (2025)

## Built on by (most cited first, 15 of 15)

- [SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents](https://mlsanity.com/paper/2605.21384/) (2026)
- [SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?](https://mlsanity.com/paper/2606.07682/) (2026)
- [Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale](https://mlsanity.com/paper/2605.20744/) (2026)
- [TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents](https://mlsanity.com/paper/2606.28480/) (2026)
- [SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?](https://mlsanity.com/paper/2608.23564/) (2026)
- [What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design](https://mlsanity.com/paper/2604.28093/) (2026)
- [Learning Generalizable Behaviors for Terminal Agents](https://mlsanity.com/paper/2608.22631/) (2026)
- [Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations](https://mlsanity.com/paper/2609.19101/) (2026)
- [RH-Detect: A Unified Benchmark for Reward Hacking Detection](https://mlsanity.com/paper/2610.10947/) (2026)
- [Cheap Reward Hacking Detection](https://mlsanity.com/paper/2606.08893/) (2026)
- [Good Benchmarks](https://mlsanity.com/paper/2607.12217/) (2026)
- [A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragility](https://mlsanity.com/paper/2608.00583/) (2026)
- [Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks](https://mlsanity.com/paper/2608.22103/) (2026)
- [AuraForge: Scaling Security Supervision for Training Coding Agents](https://mlsanity.com/paper/2610.00850/) (2026)
- [Gradient Routing Localizes and Removes Unintended Behaviors in RL](https://mlsanity.com/paper/2610.17062/) (2026)

## Abstract (arXiv)

We release Terminal Wrench, a subset of 331 terminal-agent benchmark environments, copied from the popular open benchmarks that are demonstrably reward-hackable. The data set includes 3,632 hack trajectories and 2,352 legitimate baseline trajectories across three frontier models (Claude Opus 4.6, Gemini 3.1 Pro, GPT-5.4). Each entry preserves the original task definition alongside full attack trajectories that show how the verifier was bypassed. It also includes cases where the task was not solved as intended. The tasks span system administration, machine learning, software engineering, and security challenges; the exploits range from simple output spoofing to stack-frame introspection, standard-library patching, and rootkit-style binary hijacking. Crucially, these exploits are specific to each task, rather than the evaluation harness, making them harder to patch. We also present a monitorability study in which hack trajectories are sanitized or stripped of reasoning traces and then scored by an LLM judge, showing that detection degrades meaningfully when chain-of-thought is removed (AUC drops from 0.97 to 0.92). The data set is publicly available at https://github.com/few-sh/terminal-wrench.