# TraceSIR: A Multi-Agent Framework for Structured Analysis and Reporting of Agentic Execution Traces

arXiv: https://arxiv.org/abs/2603.00623 · HTML: https://mlsanity.com/paper/2603.00623/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 10 of 10)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents](https://mlsanity.com/paper/2602.16246/) (2026)
- [TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents](https://mlsanity.com/paper/2602.21230/) (2026)
- [DREAM: Deep Research Evaluation with Agentic Metrics](https://mlsanity.com/paper/2602.18940/) (2026)
- [A2P-Vis: an Analyzer-to-Presenter Agentic Pipeline for Visual Insights Generation and Reporting](https://mlsanity.com/paper/2512.22101/) (2025)
- [Exploring General-Purpose Autonomous Multimodal Agents for Pathology Report Generation](https://mlsanity.com/paper/2601.11540/) (2025)
- [Hallucination Mitigating for Medical Report Generation](https://mlsanity.com/paper/2601.15745/) (2026)
- [BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features](https://mlsanity.com/paper/2602.16006/) (2026)

## Built on by (most cited first, 2 of 2)

- [View-oriented Conversation Compiler for Agent Trace Analysis](https://mlsanity.com/paper/2603.29678/) (2026)
- [From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis](https://mlsanity.com/paper/2609.32514/) (2026)

## Abstract (arXiv)

Agentic systems augment large language models with external tools and iterative decision making, enabling complex tasks such as deep research, function calling, and coding. However, their long and intricate execution traces make failure diagnosis and root cause analysis extremely challenging. Manual inspection does not scale, while directly applying LLMs to raw traces is hindered by input length limits and unreliable reasoning. Focusing solely on final task outcomes further discards critical behavioral information required for accurate issue localization. To address these issues, we propose TraceSIR, a multi-agent framework for structured analysis and reporting of agentic execution traces. TraceSIR coordinates three specialized agents: (1) StructureAgent, which introduces a novel abstraction format, TraceFormat, to compress execution traces while preserving essential behavioral information; (2) InsightAgent, which performs fine-grained diagnosis including issue localization, root cause analysis, and optimization suggestions; (3) ReportAgent, which aggregates insights across task instances and generates comprehensive analysis reports. To evaluate TraceSIR, we construct TraceBench, covering three real-world agentic scenarios, and introduce ReportEval, an evaluation protocol for assessing the quality and usability of analysis reports aligned with industry needs. Experiments show that TraceSIR consistently produces coherent, informative, and actionable reports, significantly outperforming existing approaches across all evaluation dimensions. Our project and video are publicly available at https://github.com/SHU-XUN/TraceSIR.