mlsanityData to 14 Oct 2026

Robust, Observable, and Evolvable Agentic Systems Engineering: A Principled Framework Validated via the Fairy GUI Agent

arXiv 2509.20729 ↗First posted 25 Sep 2025cs.AI cs.HC cs.MAcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The Agentic Paradigm faces a significant Software Engineering Absence, yielding Agentic systems commonly lacking robustness, observability, and evolvability. To address these deficiencies, we propose a principled engineering framework comprising Runtime Goal Refinement (RGR), Observable Cognitive Architecture (OCA), and Evolutionary Memory Architecture (EMA). In this framework, RGR ensures robustness and intent alignment via knowledge-constrained refinement and human-in-the-loop clarification; OCA builds an observable and maintainable white-box architecture using component decoupling, logic layering, and state-control separation; and EMA employs an execution-evolution dual-loop for evolvability. We implemented and empirically validated Fairy, a mobile GUI agent based on this framework. On RealMobile-Eval, our novel benchmark for ambiguous and complex tasks, Fairy outperformed the best SoTA baseline in user requirement completion by 33.7%. Subsequent controlled experiments, human-subject studies, and ablation studies further confirmed that the RGR enhances refinement accuracy and prevents intent deviation; the OCA improves maintainability; and the EMA is crucial for long-term performance. This research provides empirically validated specifications and a practical blueprint for building reliable, observable, and evolvable Agentic AI systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 13 of its 39 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

13 of 39 references.

References · 13

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  7. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  8. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  9. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  10. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
  11. 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Taskscited by 131
  12. 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 68
1 more
  1. 2024MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task Automationcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents
  2. 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution
  3. 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security
  4. 2026ClawMobile: Rethinking Smartphone-Native Agentic Systems
  5. 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution
  6. 2025UI-Evol: Automatic Knowledge Evolving for Computer Use Agents
  7. 2025Architectures for Building Agentic AI
  8. 2026Callability Is Not Operability: Controlled Interface Interventions for LLM Agents