A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution
On the map: LLM agents and tool use → GUI and mobile agents also fits: Software engineering coding agents
Abstract (arXiv)
Large language model agents are emerging as an important new class of users of desktop applications. However, existing desktop interfaces were designed for human end users or application developers, leaving agents to interpret visual controls or discover and compose low-level software operations. This mismatch can hinder even capable agents, yet it remains unclear what interface properties make desktop software easier for agents to use. To study this question, we treat software evolution as a natural source of interface variation and introduce DeskIF-Bench, a benchmark of 255 artifact-verified tasks across six desktop applications. By holding the task, agent, and verifier fixed while varying the software release, we measure how interface changes affect agent performance. Across two models and six applications, the newest release consistently outperforms the oldest. Trajectory analysis identifies three relevant interface properties: discoverability, granularity, and observability. Guided by these properties, we develop ANIS, a framework that synthesizes agent-native interfaces from the capabilities of installed applications. On an expanded evaluation of 368 tasks across eight applications, atomic interfaces improve success over native SDK use by 3.8 and 3.5 percentage points with DeepSeek V4 Flash and Claude Haiku 4.5, respectively, while reducing online token usage by approximately 73%. Goal-level commands further raise success to 74.5% and 67.4%, respectively.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 32 references that are in the map. First posted on arXiv 11 Oct 2026.
Builds on
28 of 32 references.
Closest ancestors · 13
Same direction, newest first
- 2026GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents22 Jun
- 2026CLI-Anything: Towards Agent-Native Computer Use2 Jun
- 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agents19 May
- 2025From Imperative to Declarative: Towards LLM-friendly OS Interfaces for Boosted Computer-Use Agents6 Oct
- 2025MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents8 Sep
- 2025API Agents vs. GUI Agents: Divergence and Convergence14 Mar
- 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Users4 Mar
- 2025WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point12 Feb
- 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
- 2024AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agents25 Sep
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
1 more
Other references · 15
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2024Beyond Browsing: API-Based Web Agentscited by 78
- 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
- 2025UFO2: The Desktop AgentOScited by 41
- 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architecturescited by 18
- 2026The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluationcited by 12
- 2025A Framework for Testing and Adapting REST APIs as LLM Toolscited by 10
- 2026Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Surveycited by 10
- 2026Schema First Tool APIs for LLM Agents: A Controlled Study of Tool Misuse, Recovery, and Budgeted Performancecited by 7
- 2025Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentationcited by 5
- 2025Automated Creation and Enrichment Framework for Improved Invocation of Enterprise APIs as Toolscited by 5
3 more
- 2026Terminal Is All You Need: Design Properties for Human-AI Agent Collaborationcited by 3
- 2026Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflowscited by 2
- 2026E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarioscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026From Human Interfaces to Agent Interfaces: Rethinking Software Design in the Age of AI-Native Systems
- 2026Software as Content: Dynamic Applications as the Human-Agent Interaction Layer
- 2026Affora: A Design System for Agent-Friendly Interfaces
- 2026Computer-Using World Model
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior
- 2026From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents
- 2026An Empirical Study of Downstream Adaptation for Agent Skills
- 2026What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces?