Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification
Not on the map. Why?
Abstract (arXiv)
Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. Yet how trustworthy such self-verification is remains poorly understood. To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent's subsequent verification and recovery behavior. Applying it to ten terminal agents on TerminalBench2.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61.43% of incorrect candidates are detected and only 49.36% of detected errors are successfully repaired. These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors. Motivated by these findings, we propose Student-Conditioned Verification Distillation (SCVD), which lets the student first produce a candidate solution and distills a stronger teacher's subsequent verification and recovery from the same interaction context. Across three Qwen3.5 backbones, SCVD improves textscPass@1 on TerminalBench2.1 by 9.74--16.85 percentage points over the corresponding base models and by 4.49--8.61 points over the standard full-trajectory distillation, while avoiding the pronounced out-of-distribution degradation of full-trajectory distillation on SWE-bench Verified.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 55 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
27 of 55 references.
References · 27
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023Teaching Large Language Models to Self-Debugcited by 988
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
- 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
- 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
- 2023FireAct: Toward Language Agent Fine-tuningcited by 256
- 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 39
15 more
- 2026Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environmentscited by 33
- 2026TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agentscited by 29
- 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 21
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 18
- 2025Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMscited by 14
- 2026Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agentscited by 14
- 2025Structured Agent Distillation for Large Language Modelcited by 13
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectoriescited by 13
- 2026SETA: Scaling Environments for Terminal Agentscited by 12
- 2026Scaling Agentic Verifier for Competitive Codingcited by 7
- 2026SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Reviewcited by 5
- 2026What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agentscited by 5
- 2026SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?cited by 4
- 2026Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixescited by 3
- 2026Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Frameworkcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Terminal Agents: A Survey of AI Agents in Command-Line Environments
- 2025SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents
- 2026Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents
- 2026Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents
- 2026FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis
- 2026Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents
- 2026SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
- 2026ReTerm: Recovery Training for Self-Improving Terminal Agents