SteerBench-Work: A Benchmark for Agent Steering at Action Boundaries
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Long-running LLM agents act through tools, and a single step can send an email, merge a pull request, or wire a payment. The steering decision is the pre-commit choice at that boundary: proceed, or hold for human or policy review. We introduce SteerBench-Work, an incident-anchored, bidirectional benchmark for that decision in workplace agents across developer operations, customer service, finance, legal, medical, HR, and security. Release v2026-05 contains 106 scenarios anchored in public incidents, paired evidence-reversed mirrors, and calibration controls, with labels split nearly evenly between proceed and hold so the two error directions get near-identical numbers of chances. A model sees the proposed action and the available evidence, returns a gate decision, and is scored on whether it crosses or holds the boundary correctly. Across 30 model conditions the failures run almost entirely in one direction: models wrongly hold authorized, evidence-cleared work on 28.1% of opportunities and wrongly allow unsafe work on 1.0%. The hardest cases are risk-resolved commits, where signed or structured evidence has already cleared a real risk trigger, and models score markedly worse on evidence-reversed mirrors of famous incidents (63.8%) than on the incidents themselves (98.5%). General capability is not the same as steering calibration: higher-capability models often over-refuse at the commit boundary, and more reasoning can repair a weak gate while leaving a calibrated one flat. The public leaderboard is at steerbench.com.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 12 of its 24 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
12 of 24 references.
References · 12
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
- 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
- 2024Rule Based Rewards for Language Model Safetycited by 108
- 2020Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Databasecited by 96
- 2025A Benchmark for Evaluating Outcome-Driven Constraint Violations in Autonomous AI Agentscited by 13
- 2026Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Modecited by 10
- 2026Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructionscited by 10
- 2026The Autonomy Tax: Defense Training Breaks LLM Agentscited by 9
- 2026Insuring Every Action: An Authority Frontier Framework for Runtime Actuarial Control of Autonomous AI Agentscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting
- 2026From Uncertainty to Action: Learning to Steer LLM Agents
- 2026ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents
- 2025SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives
- 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
- 2026AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories
- 2026AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
- 2026EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions