# CodeTracer: Towards Traceable Agent States

arXiv: https://arxiv.org/abs/2604.11641 · HTML: https://mlsanity.com/paper/2604.11641/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://mlsanity.com/paper/2412.06559/) (2024)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories](https://mlsanity.com/paper/2506.18824/) (2025)
- [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026)
- [Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents](https://mlsanity.com/paper/2510.25694/) (2025)

## Built on by (most cited first, 16 of 16)

- [Failure as a Process: An Anatomy of CLI Coding Agent Trajectories](https://mlsanity.com/paper/2607.09510/) (2026)
- [MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems](https://mlsanity.com/paper/2605.28732/) (2026)
- [Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories](https://mlsanity.com/paper/2606.02060/) (2026)
- [What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents](https://mlsanity.com/paper/2607.06184/) (2026)
- [LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents](https://mlsanity.com/paper/2608.18398/) (2026)
- [GRADE: Graph Representation of LLM Agent Dependency and Execution](https://mlsanity.com/paper/2606.22741/) (2026)
- [Locating Hidden Failures Makes Long-Horizon Agents More Reliable](https://mlsanity.com/paper/2609.17930/) (2026)
- [VERSE: Verified Self-Evolving Optimizer for Agent Harnesses](https://mlsanity.com/paper/2610.02616/) (2026)
- [Property-Level Reconstructability of Agent Decisions: An Anchor-Level Pilot Across Vendor SDK Adapter Regimes](https://mlsanity.com/paper/2605.12078/) (2026)
- [FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model](https://mlsanity.com/paper/2609.23808/) (2026)
- [From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis](https://mlsanity.com/paper/2609.32514/) (2026)
- [Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents](https://mlsanity.com/paper/2610.03634/) (2026)
- [AgentSpy: Making AI Agent Behavior Observable](https://mlsanity.com/paper/2610.06001/) (2026)
- [Trajectory-Guided Fault Localization for Agent Skill Evolution](https://mlsanity.com/paper/2610.11858/) (2026)
- [Socratic Skill Learning: Interleaving Teaching and Practice for Transferable Agent Skills](https://mlsanity.com/paper/2610.17860/) (2026)
- [SWEPatchTest: Evaluating Software Tests Beyond Reproduction](https://mlsanity.com/paper/2610.18004/) (2026)

## Abstract (arXiv)

Code agents are advancing rapidly, but debugging them is becoming increasingly difficult. As frameworks orchestrate parallel tool calls and multi-stage workflows over complex tasks, making the agent's state transitions and error propagation hard to observe. In these runs, an early misstep can trap the agent in unproductive loops or even cascade into fundamental errors, forming hidden error chains that make it hard to tell when the agent goes off track and why. Existing agent tracing analyses either focus on simple interaction or rely on small-scale manual inspection, which limits their scalability and usefulness for real coding workflows. We present CodeTracer, a tracing architecture that parses heterogeneous run artifacts through evolving extractors, reconstructs the full state transition history as a hierarchical trace tree with persistent memory, and performs failure onset localization to pinpoint the failure origin and its downstream chain. To enable systematic evaluation, we construct CodeTraceBench from a large collection of executed trajectories generated by four widely used code agent frameworks on diverse code tasks (e.g., bug fixing, refactoring, and terminal interaction), with supervision at both the stage and step levels for failure localization. Experiments show that CodeTracer substantially outperforms direct prompting and lightweight baselines, and that replaying its diagnostic signals consistently recovers originally failed runs under matched budgets. Our code and data are publicly available.