# AgentGUI: An Interface for Observing and Steering Long-Running AI Agents

arXiv: https://arxiv.org/abs/2607.26300 · HTML: https://mlsanity.com/paper/2607.26300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding](https://mlsanity.com/paper/2501.18362/) (2025)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [Interactive Debugging and Steering of Multi-Agent AI Systems](https://mlsanity.com/paper/2503.02068/) (2025)
- [Magentic-UI: Towards Human-in-the-loop Agentic Systems](https://mlsanity.com/paper/2507.22358/) (2025)
- [Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents](https://mlsanity.com/paper/2502.16069/) (2025)
- [AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous Systems](https://mlsanity.com/paper/2402.08995/) (2024)
- [ResearStudio: A Human-Intervenable Framework for Building Controllable Deep-Research Agents](https://mlsanity.com/paper/2510.12194/) (2025)

## Abstract (arXiv)

AI agents are increasingly adept at tackling complex, long-running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human-centered interfacing. Aiming to address this, we introduce AgentGUI, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open-source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p = 0.023). In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B--9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent-gui-project.github.io) and open-source repository (https://github.com/eth-medical-ai-lab/agent-gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).