# Confident and Wrong: Silent Semantic Failures in Coding Agents

arXiv: https://arxiv.org/abs/2603.25764 · HTML: https://mlsanity.com/paper/2603.25764/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 11 of 11)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)

## Built on by (most cited first, 3 of 3)

- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [The Refusal Residue: When Probes Catch Alignment Faking and When They Don't](https://mlsanity.com/paper/2607.13346/) (2026)
- [ROZA Graphs: Self-Improving Near-Deterministic RAG through Evidence-Centric Feedback](https://mlsanity.com/paper/2604.07595/) (2026)

## Abstract (arXiv)

As coding agents move into production workflows, teams need to know not only whether an agent completes a task, but whether its action can be trusted. We show that completion and trustworthiness diverge sharply and systematically. Across 1,750 trajectories on 50 SWE-bench Verified tasks, we compare four frontier models over repeated runs and separate submit rate from test-verified resolve rate. GPT-5 submits a patch on 100% of runs but resolves only 44%; Llama 4 submits on 99% but resolves 18%; and Gemini, despite submitting least often at 70%, resolves more tasks than GPT-5 (50% versus 44%). These gaps are not random: they concentrate in one dangerous failure mode we call silent semantic failure. Qualitatively, on a buggy task the agent submits a plausible-looking patch on all five runs, yet none pass, the same misinterpretation repeated rather than random error. Quantitatively, it dominates failure, covering 80% of Llama 4's failing runs and 68% of GPT-5's, and it is invisible: the outcomes are confidently and consistently wrong, so completion-based and consistency-based monitoring both look healthy exactly when the agent should not be trusted. Lightweight pre-edit prompts do not close the gap. A second probe isolates the instinct to act: given an already-fixed bug, where the right move is to abstain, most models still edit the correct code. This action bias, acting when no action is warranted, is exactly what completion metrics reward. The throughline is measurement: submit rate captures action, but trust requires validity. So evaluation must catch up: score agents by test-verified correctness over repeated runs, report its uncertainty, and reward those that know when not to act.