# BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

arXiv: https://arxiv.org/abs/2609.11028 · HTML: https://mlsanity.com/paper/2609.11028/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 75 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 25 of 25)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models](https://mlsanity.com/paper/2201.03544/) (2022)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [AI Safety Gridworlds](https://mlsanity.com/paper/1711.09883/) (2017)
- [SpecRover: Code Intent Extraction via LLMs](https://mlsanity.com/paper/2408.02232/) (2024)
- [Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows](https://mlsanity.com/paper/2605.27922/) (2026)
- [ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces](https://mlsanity.com/paper/2604.05172/) (2026)
- [Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use](https://mlsanity.com/paper/2605.02964/) (2026)
- [EvilGenie: A Reward Hacking Benchmark](https://mlsanity.com/paper/2511.21654/) (2025)
- [Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?](https://mlsanity.com/paper/2510.05244/) (2025)
- [Identifying Patch Correctness in Test-Based Program Repair](https://mlsanity.com/paper/1706.09120/) (2017)
- [Detecting Safety Violations Across Many Agent Traces](https://mlsanity.com/paper/2604.11806/) (2026)
- [Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack](https://mlsanity.com/paper/2605.12673/) (2026)
- [Automated Benchmark Auditing for AI Agents and Large Language Models](https://mlsanity.com/paper/2605.26079/) (2026)
- [RewardHackingAgents: Benchmarking Evaluation Integrity for LLM ML-Engineering Agents](https://mlsanity.com/paper/2603.11337/) (2026)
- [Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale](https://mlsanity.com/paper/2605.20744/) (2026)
- [A Survey of Challenges for Runtime Verification from Advanced Application Domains (Beyond Software)](https://mlsanity.com/paper/1811.06740/) (2018)
- [TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling](https://mlsanity.com/paper/2605.27690/) (2026)
- [Capability Gates Are Not Authorization: Confused-Deputy Failures in LLM Agent Frameworks](https://mlsanity.com/paper/2606.28679/) (2026)
- [Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents](https://mlsanity.com/paper/2604.27233/) (2026)
- [Lean4Agent: Formal Modeling and Verification for Agent Workflow and Trajectory](https://mlsanity.com/paper/2606.06523/) (2026)
- [Let it Flow: A Formally Verified Compilation Framework for Asynchronous Dataflow](https://mlsanity.com/paper/2608.05451/) (2026)

## Built on by (most cited first, 1 of 1)

- [CWE-bench: Refocusing Agentic Security from Exploitation to Defense](https://mlsanity.com/paper/2610.17020/) (2026)

## Abstract (arXiv)

LM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifacts, and receive rewards from outcome procedures. This interactivity makes evaluations vulnerable to reward hacking: an agent improves its measured score by exploiting the reward-relevant trajectory instead of solving the intended task. Existing defenses rely largely on task-specific patches, prompt instructions, or post-hoc detectors. They do not provide reusable evidence that a concrete run remained within its intended evaluation boundary. This paper presents BenchShield, a model-backed instrumentation layer for reward integrity in LLM-agent evaluation. BenchShield grounds detection in a finite lifecycle model of an evaluation's reward-relevant events. Within the benchmark infrastructure, two complementary analyses operate over this model. A static, phase-aware taint analysis exposes reward-hacking paths before a run. Its runtime counterpart uses infrastructure-side evidence to attribute concrete agent use and emit evidence-backed claims. We construct BenchShield Trajectories, a human-labeled corpus of 456 adjudicated trajectories from more than 31,000 public agent runs across three benchmarks. Compared with an agentic hackability scanner baseline on the same tasks and model, BenchShield improves full-chain recall from 23-94% to 77-100%, same-vector coverage from 16-56% to 43-78%, and reduces per-task cost by up to 65%. Its runtime analysis achieves 96% accuracy in detecting reward hacking from infrastructure-side evidence.