Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks also fits: Reinforcement learning for LLM agents
Abstract (arXiv)
The rapid progression of large language models is extending AI from passive content generation into the active workflows of engineering and scientific discovery. This shift raises a compelling question: can AI be both the object of development and an active participant in building next-generation AI systems? We explore this question by building Qwen-Planner-Agent within a closed-loop AI-for-AI framework for scalable development and iterative improvement. Mobile planning offers a demanding test of this approach: complex, long-horizon tasks challenge agent reliability, while costly real-device interaction limits development scalability. The framework connects data production, model training, and deployment through a shared action-feedback-verification contract. (i) AI for Data builds a human-gated agentic data flywheel in which specialized agents construct tasks, collect interaction trajectories, curate and balance training data, and use training feedback to guide subsequent data generation. (ii) AI for Training combines a supervised planning cold start with hybrid-environment online agentic reinforcement learning, where we introduce Competence-Aware Reward-and-Advantage Engineering (CARE) to reduce reasoning and tool-use costs while preserving task performance. (iii) AI drives model--harness co-evolution through an execution-evidence-driven loop that orchestrates memory, skills, and tools at runtime and feeds structured action feedback and preserved failure traces back into coordinated model and harness adaptation. Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination. Further evaluations of our model show improvements across non-mobile agentic benchmarks while largely preserving general capabilities.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 53 of its 66 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
53 of 66 references.
Closest ancestors · 3
Same direction, newest first
- 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents7 Apr
- 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution29 Oct
- 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML3 Oct
Other references · 50
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
38 more
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2023Large Language Models as Optimizerscited by 830
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
- 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
- 2024Agent Workflow Memorycited by 342
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
- 2024TextGrad: Automatic "Differentiation" via Textcited by 250
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
- 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
- 2024Are We Done with MMLU?cited by 178
- 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 175
- 2025Agentic Reinforced Policy Optimizationcited by 166
- 2026Towards End-to-End Automation of AI Researchcited by 150
- 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Taskscited by 128
- 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 94
- 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
- 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
- 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 63
- 2026Natural-Language Agent Harnessescited by 54
- 2025Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMscited by 52
- 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Serverscited by 46
- 2025MoM: Linear Sequence Modeling with Mixture-of-Memoriescited by 44
- 2025MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agentscited by 43
- 2025MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Usecited by 37
- 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 29
- 2025TeleMem: Building Long-Term and Multimodal Memory for Agentic AIcited by 14
- 2026MemoHarness: Agent Harnesses That Learn from Experiencecited by 14
- 2026SkillDroid: Compile Once, Reuse Forevercited by 4
- 2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
- 2026MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Taskscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
- 2025Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
- 2025A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyond
- 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
- 2026AI Agent Systems: Architectures, Applications, and Evaluation
- 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
- 2026Atria Dawn: The Dawn of Agentic Superintelligence
- 2026AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use