mlsanityData to 7 Oct 2026

GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL

arXiv 2602.22190 ↗First posted 25 Feb 2026cs.LG cs.AI cs.CLcited by 14

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Open-source native GUI agents still lag behind closed-source systems on long-horizon navigation tasks. This gap stems from two limitations: a shortage of high-quality, action-aligned reasoning data, and the direct adoption of generic post-training pipelines that overlook the unique challenges of GUI agents. We identify two fundamental issues in these pipelines: (i) standard SFT with CoT reasoning often hurts grounding, and (ii) step-wise RLVR-tyle training faces partial verifiability, where multiple actions can be correct but only a single demonstrated action is used for verification. This makes offline step-wise metrics weak predictors of online task success. In this work, we present GUI-Libra, a tailored training recipe that addresses these challenges. First, to mitigate the scarcity of action-aligned reasoning data, we introduce a data construction and filtering pipeline and release a curated 81K GUI reasoning dataset. Second, to reconcile reasoning with grounding, we propose action-aware SFT that mixes reasoning-then-action and direct-action data and reweights tokens to emphasize action and grounding. Third, to stabilize RL under partial verifiability, we identify the overlooked importance of KL regularization in RLVR and show that a KL trust region is critical for improving offline-to-online predictability; we further introduce success-adaptive scaling to downweight unreliable negative gradients. Across diverse web and mobile benchmarks, GUI-Libra consistently improves both step-wise accuracy and end-to-end task completion. Our results suggest that carefully designed post-training and data curation can unlock significantly stronger task-solving capabilities without costly online data collection. We release our dataset, code, and models to facilitate further research on data-efficient post-training for reasoning-capable GUI agents.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 52 of its 84 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

52 of 84 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec
  2. 2025WebSTAR: Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering22 Nov
  3. 2025GUI-360^circ: A Comprehensive Dataset and Benchmark for Computer-Using Agents6 Nov
  4. 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning23 Oct
  5. 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents30 Sep
  6. 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data18 Sep
  7. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  8. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
  9. 2025Phi-Ground Tech Report: Advancing Perception in GUI Grounding31 Jul
  10. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents3 Jun
  11. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis19 May
  12. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
8 more
  1. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
  2. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  3. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis27 Dec
  4. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  5. 2024AutoGLM: Autonomous Foundation Agents for GUIs28 Oct
  6. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
  7. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
  8. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 32

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2025Group Sequence Policy Optimizationcited by 788
  8. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  9. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  10. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  11. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  12. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
20 more
  1. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  2. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  3. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  4. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  5. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
  6. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  7. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  8. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  9. 2025GTA1: GUI Test-time Scaling Agentcited by 102
  10. 2025Agent Learning via Early Experiencecited by 79
  11. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 60
  12. 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agentscited by 60
  13. 2025Fara-7B: An Efficient Agentic Model for Computer Usecited by 34
  14. 2023Aligning Large Language Models with Human: A Surveycited by 31
  15. 2025Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Modelscited by 30
  16. 2025InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimizationcited by 24
  17. 2026WebGym: Scaling Training Environments for Visual Web Agents with Realistic Taskscited by 19
  18. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learningcited by 10
  19. 2025BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learningcited by 7
  20. 2025PixelWeb: The First Web GUI Dataset with Pixel-Wise Labelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners
  2. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
  3. 2026LocalLSTC: A Long Short-Term Control Architecture for Locally Deployed GUI Agents
  4. 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation
  5. 2025NaturalGAIA: A Verifiable Benchmark and Hierarchical Framework for Long-Horizon GUI Tasks
  6. 2026Learning Reliable GUI Agents under Imperfect Priors
  7. 2025GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks
  8. 2026UI-Venus-2 Technical Report