# ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

arXiv: https://arxiv.org/abs/2606.19380 · HTML: https://mlsanity.com/paper/2606.19380/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 69 references that are in the map.
- First posted on arXiv 13 Jun 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations](https://mlsanity.com/paper/2411.00640/) (2024)
- [SafeArena: Evaluating the Safety of Autonomous Web Agents](https://mlsanity.com/paper/2503.04957/) (2025)
- [RedCode: Risky Code Execution and Generation Benchmark for Code Agents](https://mlsanity.com/paper/2411.07781/) (2024)
- [Towards a Science of AI Agent Reliability](https://mlsanity.com/paper/2602.16666/) (2026)
- [Contextual Agent Security: A Policy for Every Purpose](https://mlsanity.com/paper/2501.17070/) (2025)
- [Operationalizing Contextual Integrity in Privacy-Conscious Assistants](https://mlsanity.com/paper/2408.02373/) (2024)
- [Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety](https://mlsanity.com/paper/2510.16492/) (2025)
- [The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?](https://mlsanity.com/paper/2601.23045/) (2026)
- [StereoMath: An Accessible and Musical Equation Editor](https://mlsanity.com/paper/2501.01404/) (2025)
- [Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor](https://mlsanity.com/paper/2508.07576/) (2025)

## Abstract (arXiv)

Widespread deployment of AI agents in software engineering is surfacing a long tail of rare but highly dangerous misalignment bugs. Since sampling this behavior is intractable, we decompose these failures into three distinct mechanisms: underspecification, where default model behavior is unsafe; capability errors, where the safe action is available but the model does not adhere to it; and agent harness errors, where the safe action fails to execute. Across 8 evaluations that stress test these mechanisms, we find that frontier models are difficult to steer, elicit dangerous behavior through just 3 conditioning examples, and can randomly generate destructive commands at nontrivial probabilities. These three mechanisms naturally lend to targeted mitigations for each one, inspiring our harness modification ClayBuddy. ClayBuddy allows the agent to modify its own harness to provide safety guarantees, including a novel tool to selectively truncate its own context. We measure ClayBuddy's efficacy across safety and capabilities, finding that it eliminates violations across some tasks. This demonstrates that our decomposition is effective and is an example of our harness design philosophy.