Robust, Observable, and Evolvable Agentic Systems Engineering: A Principled Framework Validated via the Fairy GUI Agent
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
The Agentic Paradigm faces a significant Software Engineering Absence, yielding Agentic systems commonly lacking robustness, observability, and evolvability. To address these deficiencies, we propose a principled engineering framework comprising Runtime Goal Refinement (RGR), Observable Cognitive Architecture (OCA), and Evolutionary Memory Architecture (EMA). In this framework, RGR ensures robustness and intent alignment via knowledge-constrained refinement and human-in-the-loop clarification; OCA builds an observable and maintainable white-box architecture using component decoupling, logic layering, and state-control separation; and EMA employs an execution-evolution dual-loop for evolvability. We implemented and empirically validated Fairy, a mobile GUI agent based on this framework. On RealMobile-Eval, our novel benchmark for ambiguous and complex tasks, Fairy outperformed the best SoTA baseline in user requirement completion by 33.7%. Subsequent controlled experiments, human-subject studies, and ablation studies further confirmed that the RGR enhances refinement accuracy and prevents intent deviation; the OCA improves maintainability; and the EMA is crucial for long-term performance. This research provides empirically validated specifications and a practical blueprint for building reliable, observable, and evolvable Agentic AI systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 13 of its 39 references that are in the map. First posted on arXiv 25 Sep 2025.
Builds on
13 of 39 references.
References · 13
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
- 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
- 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Taskscited by 131
- 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 68
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents
- 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution
- 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security
- 2026ClawMobile: Rethinking Smartphone-Native Agentic Systems
- 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution
- 2025UI-Evol: Automatic Knowledge Evolving for Computer Use Agents
- 2025Architectures for Building Agentic AI
- 2026Callability Is Not Operability: Controlled Interface Interventions for LLM Agents