# InquireMobile: Teaching VLM-based Mobile Agent to Request Human Assistance via Reinforcement Fine-Tuning

arXiv: https://arxiv.org/abs/2508.19679 · HTML: https://mlsanity.com/paper/2508.19679/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents](https://mlsanity.com/paper/2504.10458/) (2025)
- [GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices](https://mlsanity.com/paper/2406.08451/) (2024)
- [Mobile-Agent-v3: Fundamental Agents for GUI Automation](https://mlsanity.com/paper/2508.15144/) (2025)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model](https://mlsanity.com/paper/2503.05132/) (2025)
- [InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners](https://mlsanity.com/paper/2504.14239/) (2025)
- [Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks](https://mlsanity.com/paper/2501.11733/) (2025)
- [AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents](https://mlsanity.com/paper/2407.17490/) (2024)
- [AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents](https://mlsanity.com/paper/2410.24024/) (2024)
- [GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents](https://mlsanity.com/paper/2505.15810/) (2025)
- [Understanding the Weakness of Large Language Model Agents within a Complex Android Environment](https://mlsanity.com/paper/2402.06596/) (2024)
- [Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents](https://mlsanity.com/paper/2407.00993/) (2024)
- [AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.01391/) (2025)
- [Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction](https://mlsanity.com/paper/2305.08144/) (2023)
- [OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agents](https://mlsanity.com/paper/2503.16465/) (2025)
- [MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions](https://mlsanity.com/paper/2502.16796/) (2025)
- [GUI-Robust: A Comprehensive Dataset for Testing GUI Agent Robustness in Real-World Anomalies](https://mlsanity.com/paper/2506.14477/) (2025)

## Built on by (most cited first, 6 of 6)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows](https://mlsanity.com/paper/2510.24411/) (2025)
- [Say One Thing, Do Another? Diagnosing Reasoning-Execution Gaps in VLM-Powered Mobile-Use Agents](https://mlsanity.com/paper/2510.02204/) (2025)
- [AndroidLens: Long-latency Evaluation with Nested Sub-targets for Android GUI Agents](https://mlsanity.com/paper/2512.21302/) (2025)
- [GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents](https://mlsanity.com/paper/2605.20246/) (2026)
- [Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agents](https://mlsanity.com/paper/2605.28629/) (2026)

## Abstract (arXiv)

Recent advances in Vision-Language Models (VLMs) have enabled mobile agents to perceive and interact with real-world mobile environments based on human instructions. However, the current fully autonomous paradigm poses potential safety risks when model understanding or reasoning capabilities are insufficient. To address this challenge, we first introduce InquireBench, a comprehensive benchmark specifically designed to evaluate mobile agents' capabilities in safe interaction and proactive inquiry with users, encompassing 5 categories and 22 sub-categories, where most existing VLM-based agents demonstrate near-zero performance. In this paper, we aim to develop an interactive system that actively seeks human confirmation at critical decision points. To achieve this, we propose InquireMobile, a novel model inspired by reinforcement learning, featuring a two-stage training strategy and an interactive pre-action reasoning mechanism. Finally, our model achieves an 46.8% improvement in inquiry success rate and the best overall success rate among existing baselines on InquireBench. We will open-source all datasets, models, and evaluation codes to facilitate development in both academia and industry.