InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
The emergence of Multimodal Large Language Models (MLLMs) has propelled the development of autonomous agents that operate on Graphical User Interfaces (GUIs) using pure visual input. A fundamental challenge is robustly grounding natural language instructions. This requires a precise spatial alignment, which accurately locates the coordinates of each element, and, more critically, a correct semantic alignment, which matches the instructions to the functionally appropriate UI element. Although Reinforcement Learning with Verifiable Rewards (RLVR) has proven to be effective at improving spatial alignment for these MLLMs, we find that inefficient exploration bottlenecks semantic alignment, which prevent models from learning difficult semantic associations. To address this exploration problem, we present Adaptive Exploration Policy Optimization (AEPO), a new policy optimization framework. AEPO employs a multi-answer generation strategy to enforce broader exploration, which is then guided by a theoretically grounded Adaptive Exploration Reward (AER) function derived from first principles of efficiency eta=U/C. Our AEPO-trained models, InfiGUI-G1-3B and InfiGUI-G1-7B, establish new state-of-the-art results across multiple challenging GUI grounding benchmarks, achieving significant relative improvements of up to 9.0% against the naive RLVR baseline on benchmarks designed to test generalization and semantic understanding. Resources are available at https://github.com/InfiXAI/InfiGUI-G1.
Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 39 references that are in the map. First posted on arXiv 7 Aug 2025.
Builds on
31 of 39 references.
Closest ancestors · 11
Same direction, newest first
- 2025UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding29 Jul
- 2025R-VLM: Region-Aware Vision Language Model for Precise GUI Grounding8 Jul
- 2025ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding30 Jun
- 2025GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior9 Jun
- 2025ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search21 May
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
- 2025UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesis15 Apr
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 20
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
- 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
- 2025GTA1: GUI Test-time Scaling Agentcited by 106
8 more
- 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 95
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
- 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
- 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agentscited by 67
- 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
- 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoningcited by 31
- 2025Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Modelscited by 10
Built on by
Citing papers per year: 2025: 8; 2026: 19.
27 papers.
Sort:
By area:
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
- 2025Grounding Computer Use Agents on Human Demonstrationscited by 22
- 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
- 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RLcited by 16
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
- 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Groundingcited by 8
- 2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Groundingcited by 8
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 5
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agentscited by 5
- 2025See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Togglescited by 4
- 2026POINTS-GUI-G: GUI-Grounding Journeycited by 4
15 more
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoningcited by 4
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Groundingcited by 3
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agentscited by 3
- 2025How Auxiliary Reasoning Unleashes GUI Grounding in VLMscited by 2
- 2026WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environmentscited by 2
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Groundingcited by 2
- 2026The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agentscited by 2
- 2026SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesiscited by 0
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agentscited by 0
- 2026Covering Human Action Space for Computer Use: Data Synthesis and Benchmarkcited by 0
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Groundingcited by 0
- 2026Recurrent Latent Visual Search for GUI Groundingcited by 0
- 2026ConClick: Element-Wise Contrastive Learning Improves GUI Groundingcited by 0
- 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learningcited by 0
- 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning13 Oct 2026
- 2026ConClick: Element-Wise Contrastive Learning Improves GUI Grounding9 Oct 2026
- 2026Recurrent Latent Visual Search for GUI Grounding4 Oct 2026
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding14 Sep 2026
- 2026The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agents6 Aug 2026
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents7 Jul 2026
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding29 Jun 2026
- 2026WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments13 May 2026
- 2026Covering Human Action Space for Computer Use: Data Synthesis and Benchmark12 May 2026
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding4 May 2026
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitants30 Apr 2026
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents27 Apr 2026
15 more
- 2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding15 Apr 2026
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning8 Apr 2026
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents25 Mar 2026
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements15 Mar 2026
- 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL25 Feb 2026
- 2026POINTS-GUI-G: GUI-Grounding Journey6 Feb 2026
- 2026SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis26 Jan 2026
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec 2025
- 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills18 Dec 2025
- 2025Grounding Computer Use Agents on Human Demonstrations10 Nov 2025
- 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding2 Nov 2025
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning23 Oct 2025
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents19 Sep 2025
- 2025See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles17 Sep 2025
- 2025How Auxiliary Reasoning Unleashes GUI Grounding in VLMs15 Sep 2025
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
- 2025Grounding Computer Use Agents on Human Demonstrationscited by 22
- 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
- 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RLcited by 16
- 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Groundingcited by 8
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agentscited by 5
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agentscited by 3
- 2026The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agentscited by 2
- 2026SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesiscited by 0
- 2026Covering Human Action Space for Computer Use: Data Synthesis and Benchmarkcited by 0
12 most cited of 14 All 14 (JSON)
- 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning13 Oct 2026
- 2026ConClick: Element-Wise Contrastive Learning Improves GUI Grounding9 Oct 2026
- 2026The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agents6 Aug 2026
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents7 Jul 2026
- 2026Covering Human Action Space for Computer Use: Data Synthesis and Benchmark12 May 2026
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitants30 Apr 2026
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents25 Mar 2026
- 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL25 Feb 2026
- 2026SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis26 Jan 2026
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec 2025
- 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills18 Dec 2025
- 2025Grounding Computer Use Agents on Human Demonstrations10 Nov 2025
12 newest of 14 All 14 (JSON)
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 5
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Groundingcited by 3
- 2026WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environmentscited by 2
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agentscited by 0
- 2026Recurrent Latent Visual Search for GUI Groundingcited by 0
- 2026Recurrent Latent Visual Search for GUI Grounding4 Oct 2026
- 2026WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments13 May 2026
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding4 May 2026
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents27 Apr 2026
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements15 Mar 2026
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
- 2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Groundingcited by 8
- 2025See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Togglescited by 4
- 2026POINTS-GUI-G: GUI-Grounding Journeycited by 4
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Groundingcited by 2
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Groundingcited by 0
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding14 Sep 2026
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding29 Jun 2026
- 2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding15 Apr 2026
- 2026POINTS-GUI-G: GUI-Grounding Journey6 Feb 2026
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents19 Sep 2025
- 2025See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles17 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning
- 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistency
- 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
- 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay
- 2025EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forget
- 2026Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Grounding
- 2025AWPO: Enhancing Tool-Use of Large Language Models through Adaptive Integration of Reasoning Rewards
- 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding