mlsanityData to 7 Oct 2026

Learning with Challenges: Adaptive Difficulty-Aware Data Generation for Mobile GUI Agent Training

arXiv 2601.22781 ↗First posted 30 Jan 2026cs.AIcited by 4

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Large-scale, high-quality interaction trajectories are essential for advancing mobile Graphical User Interface (GUI) agents. While existing methods typically rely on labor-intensive human demonstrations or automated model exploration to generate GUI trajectories, they lack fine-grained control over task difficulty. This fundamentally restricts learning effectiveness due to the mismatch between the training difficulty and the agent's capabilities. Inspired by how humans acquire skills through progressively challenging tasks, we propose MobileGen, a novel data generation framework that adaptively aligns training difficulty with the GUI agent's capability frontier. Specifically, MobileGen explicitly decouples task difficulty into structural (e.g., trajectory length) and semantic (e.g., task goal) dimensions. It then iteratively evaluates the agent on a curated prior dataset to construct a systematic profile of its capability frontier across these two dimensions. With this profile, the probability distribution of task difficulty is adaptively computed, from which the target difficulty for the next round of training can be sampled. Guided by the sampled difficulty, a multi-agent controllable generator is finally used to synthesize high-quality interaction trajectories along with corresponding task instructions. Extensive experiments show that MobileGen consistently outperforms existing data generation methods by improving the average performance of GUI agents by 1.57 times across multiple challenging benchmarks. This highlights the importance of capability-aligned data generation for effective mobile GUI agent training.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 45 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

40 of 45 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025ColorAgent: Building A Robust, Personalized, and Interactive OS Agent22 Oct
  2. 2025AndroidControl-Curated: Revealing the True Potential of GUI Agents through Benchmark Purification21 Oct
  3. 2025Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation28 Sep
  4. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoning19 Sep
  5. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  6. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
  7. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug
  8. 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning19 Jul
  9. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Cost29 May
  10. 2025UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents27 May
  11. 2025Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents19 May
  12. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects28 Apr
8 more
  1. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  2. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis27 Dec
  3. 2024Large Language Model-Brained GUI Agents: A Survey27 Nov
  4. 2024AutoGLM: Autonomous Foundation Agents for GUIs28 Oct
  5. 2024GUICourse: From General Vision Language Models to Versatile GUI Agents17 Jun
  6. 2024On the Effects of Data Scale on UI Control Agents6 Jun
  7. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan
  8. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan

Other references · 20

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2025OpenAI GPT-5 System Cardcited by 910
  10. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  12. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
8 more
  1. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
  2. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  3. 2023DroidBot-GPT: GPT-powered UI Automation for Androidcited by 47
  4. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 41
  5. 2025UFO2: The Desktop AgentOScited by 38
  6. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  7. 2024Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platformscited by 13
  8. 2025AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesiscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark
  2. 2026Learning Reliable GUI Agents under Imperfect Priors
  3. 2026MagicGUI-RMS: A Multi-Agent Reward Model System for Self-Evolving GUI Agents via Automated Feedback Reflux
  4. 2025MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agents
  5. 2025CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks
  6. 2026Can Agents Work for Everyone? Cross-User Reliability for Mobile GUI Agents in Personalized User Interfaces
  7. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  8. 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning