NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks
Not on the map. Why?
Abstract (arXiv)
Recent advances in Graphical User Interface (GUI) and embodied navigation have driven progress, yet these domains have largely evolved in isolation, with disparate datasets and training paradigms. In this paper, we observe that both tasks can be formulated as Markov Decision Processes (MDP), suggesting a foundational principle for their unification. Hence, we present NaviMaster, the first unified agent capable of unifying GUI navigation and embodied navigation within a single framework. Specifically, NaviMaster (i) proposes a visual-target trajectory collection pipeline that generates trajectories for both GUI and embodied tasks using a single formulation. (ii) employs a unified reinforcement learning framework on the mix data to improve generalization. (iii) designs a novel distance-aware reward to ensure efficient learning from the trajectories. Through extensive experiments on out-of-domain benchmarks, NaviMaster is shown to outperform state-of-the-art agents in GUI navigation, spatial affordance prediction, and embodied navigation. Ablation studies further demonstrate the efficacy of our unified training strategy, data mixing strategy, and reward design. Our codes, data, and checkpoints are available at https://iron-boyy.github.io/navimaster-page/.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 4 Aug 2025.
Builds on
33 of 51 references.
References · 33
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 244
21 more
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 221
- 2024On the Effects of Data Scale on UI Control Agentscited by 194
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 168
- 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
- 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Webcited by 157
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasonerscited by 140
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
- 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 113
- 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 111
- 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
- 2023VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenescited by 42
- 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
- 2024LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automationcited by 36
- 2024FastLGS: Speeding up Language Embedded Gaussians with Feature Grid Mappingcited by 34
- 2025UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Groundingcited by 33
- 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligencecited by 15
- 2025GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learningcited by 11
- 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
- 2025IDMR: Towards Instance-Driven Precise Visual Correspondence in Multimodal Retrievalcited by 8
Built on by
Citing papers per year: 2025: 4; 2026: 6.
10 papers.
- 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 93
- 2025HiconAgent: History Context-aware Policy Optimization for GUI Agentscited by 13
- 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Executioncited by 13
- 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actionscited by 6
- 2025OmniActor: A Generalist GUI and Embodied Agent for 2D&3D Worldscited by 4
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimizationcited by 4
- 2026Structured Observation Language for Efficient and Generalizable Vision-Language Navigationcited by 3
- 2025SNOW: Spatio-Temporal Scene Understanding with World Knowledge for Open-World Embodied Reasoningcited by 2
- 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Modelscited by 2
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agentscited by 0
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents18 May 2026
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization15 Apr 2026
- 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions8 Apr 2026
- 2026Structured Observation Language for Efficient and Generalizable Vision-Language Navigation29 Mar 2026
- 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Models21 Feb 2026
- 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution28 Jan 2026
- 2025SNOW: Spatio-Temporal Scene Understanding with World Knowledge for Open-World Embodied Reasoning18 Dec 2025
- 2025HiconAgent: History Context-aware Policy Optimization for GUI Agents1 Dec 2025
- 2025OmniActor: A Generalist GUI and Embodied Agent for 2D&3D Worlds2 Sep 2025
- 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2025GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning
- 2026Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction
- 2026Learn from Move: the Next Step for GUI Agents
- 2026Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization
- 2026AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation
- 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
- 2026MagicGUI-RMS: A Multi-Agent Reward Model System for Self-Evolving GUI Agents via Automated Feedback Reflux
- 2025CompassNav: Steering From Path Imitation To Decision Understanding In Navigation