CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Mobile GUI agents remain brittle when deployed to applications absent from source training. We study novel-app generalization under a limited target interaction budget and without target demonstrations. We introduce CoAdapt-GUI, a test-time adaptation (TTA) framework that jointly adapts structured workflow context and policy from the agent's own target-app rollouts and rewards. The workflow context retains transferable procedures, failure modes, and verification rules while excluding app-bound source details. This separation allows reusable workflow knowledge to guide adaptation without transferring source-interface state. For policy adaptation, task-context-matched group-relative optimization updates a LoRA adapter on a frozen vision-language model. Across two unseen-app evaluations, CoAdapt-GUI reaches 45.0% on AndroidWorld-Generalization, compared with 37.5% for the reported Policy-Only TTA baseline, and raises AndroidWorld Plus performance from 38.6% to 52.9%. These results show that transfer-constrained workflow context provides substantial gains and that joint policy adaptation further improves held-out performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 28 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
19 of 28 references.
Closest ancestors · 12
Same direction, newest first
- 2026OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis16 Apr
- 2025MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agents10 Sep
- 2025Agent-SAMA: State-Aware Mobile Assistant29 May
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
- 2024AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents31 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2024On the Effects of Data Scale on UI Control Agents6 Jun
- 2024Benchmarking Mobile Device Control Agents across Diverse Configurations25 Apr
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
- 2023AppAgent: Multimodal Agents as Smartphone Users21 Dec
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 7
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 20
- 2026Evolutionary System Prompt Learning for Reinforcement Learning in LLMscited by 5
- 2025Beyond Training: Enabling Self-Evolution of Agents with MOBIMEMcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization
- 2026Same Tasks, Different Apps: Why Mobile GUI Agents Fail to Generalize?
- 2026JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
- 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution
- 2026APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
- 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
- 2025AndroidLens: Long-latency Evaluation with Nested Sub-targets for Android GUI Agents
- 2025GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration