Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
The paradigm of agentic AI is shifting from engineered complex workflows to post-training native models. However, existing agents are typically confined to static, predefined action spaces-such as exclusively using APIs, GUI events, or robotic commands. This rigidity limits their adaptability in dynamic environments where the optimal granularity of interaction varies contextually. To bridge this gap, we propose CrossHA, a unified agentic model that masters heterogeneous action spaces and autonomously selects the most effective interface for each step of a trajectory. We introduce a comprehensive training pipeline that integrates cold-start supervised fine-tuning with a Multi-Turn Group Relative Policy Optimization (GRPO) algorithm. This approach enables the agent to learn adaptive action switching-balancing high-level efficiency with low-level precision-without human-specified rules. Extensive experiments on over 800 tasks in the open-world Minecraft environment demonstrate that CrossHA achieves state-of-the-art performance. By dynamically leveraging the strengths of diverse action spaces, our model significantly outperforms fixed-action baselines, exhibiting superior generalization and efficiency in long-horizon reasoning. All code and models are available at https://github.com/CraftJarvis/OpenHA.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 49 references that are in the map. First posted on arXiv 10 Dec 2025.
Builds on
42 of 49 references.
Closest ancestors · 5
Same direction, newest first
- 2025Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents27 Oct
- 2025OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft13 Sep
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
- 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models10 Nov
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun
Other references · 37
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2017Neural Discrete Representation Learningcited by 4,712
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
25 more
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
- 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
- 2025Seed1.5-VL Technical Reportcited by 373
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2022Behavior Transformers: Cloning k modes with one stonecited by 304
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
- 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
- 2024RT-H: Action Hierarchies Using Languagecited by 210
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
- 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
- 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 107
- 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 97
- 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
- 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
- 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
- 2025MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agentscited by 38
- 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 35
- 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agentscited by 29
- 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 28
- 2024ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Promptingcited by 20
- 2025Open-World Skill Discovery from Unsegmented Demonstrationscited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
- 2025AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
- 2026MOSAIC: A Universal Agent-Level Interface for Cross-Paradigm Agent Mixing and Human-AI Collaboration
- 2025Training Agents Inside of Scalable World Models
- 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy
- 2025Optimus-3: Dual-Router Aligned Mixture-of-Experts Agent with Dual-Granularity Reasoning-Aware Policy Optimization
- 2026When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
- 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem