# ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution

arXiv: https://arxiv.org/abs/2604.13787 · HTML: https://mlsanity.com/paper/2604.13787/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [ZeroSearch: Incentivize the Search Capability of LLMs without Searching](https://mlsanity.com/paper/2505.04588/) (2025)
- [TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs](https://mlsanity.com/paper/2303.16434/) (2023)
- [SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning](https://mlsanity.com/paper/2509.02479/) (2025)
- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [ToolGen: Unified Tool Retrieval and Calling via Generation](https://mlsanity.com/paper/2410.03439/) (2024)
- [DeepAgent: A General Reasoning Agent with Scalable Toolsets](https://mlsanity.com/paper/2510.21618/) (2025)
- [Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval](https://mlsanity.com/paper/2408.01875/) (2024)
- [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://mlsanity.com/paper/2506.20923/) (2025)
- [Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent](https://mlsanity.com/paper/2505.07596/) (2025)
- [Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data](https://mlsanity.com/paper/2511.12609/) (2025)
- [Improving Tool Retrieval by Leveraging Large Language Models for Query Generation](https://mlsanity.com/paper/2412.03573/) (2024)
- [LMEB: Long-horizon Memory Embedding Benchmark](https://mlsanity.com/paper/2603.12572/) (2026)

## Built on by (most cited first, 7 of 7)

- [ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents](https://mlsanity.com/paper/2608.19662/) (2026)
- [MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Reranking](https://mlsanity.com/paper/2607.17751/) (2026)
- [SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents](https://mlsanity.com/paper/2608.18852/) (2026)
- [StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?](https://mlsanity.com/paper/2609.00787/) (2026)
- [Toolcompass: Guiding Tool Trialing, Not Suppressing It](https://mlsanity.com/paper/2609.25678/) (2026)
- [ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments](https://mlsanity.com/paper/2609.33618/) (2026)
- [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) enhance their problem-solving capability by utilizing external tools. However, in open-world scenarios with massive and evolving tool repositories, existing methods relying on static embedding retrieval or parameter memorization of tools struggle to align user intent with tool semantics or generalize to unseen tools, respectively, leading to suboptimal accuracy of open-world tool retrieval and execution. To address these, we present ToolOmni, a unified agentic framework that enables LLMs for open-world tool use by proactive retrieval and grounded execution within a reasoning loop. First, we construct a cold-start multi-turn interaction dataset to instill foundational agentic capabilities via Supervised Fine-Tuning (SFT). Then, we introduce open-world tool learning based on a Decoupled Multi-Objective GRPO algorithm, which simultaneously optimizes LLMs for both tool retrieval accuracy and execution efficacy in online environments. Extensive experiments demonstrate that ToolOmni achieves state-of-the-art performance both in retrieval and execution, surpassing strong baselines by a significant margin of +10.8% in end-to-end execution success rate, while exhibiting exceptional robustness and generalization capabilities.