mlsanityData to 7 Oct 2026

Toolcompass: Guiding Tool Trialing, Not Suppressing It

arXiv 2609.25678 ↗First posted 22 Sep 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) agents must generalize from tools seen during training to unseen tools at deployment. A key challenge is tool trialing, i.e., excessive trials waste the interaction budget, whereas selective trials enable exploration of unfamiliar tools. Existing outcome-based post-training leaves wasteful trials unguided, while turn-level supervision may suppress necessary exploration. We introduce ToolCompass, a post-training framework that guides tool trialing by organizing tool-call representations according to shared functions. Specifically, ToolCompass models each function class as a von Mises--Fisher distribution and jointly reduces intra-function variation across domains and increases inter-function separation. This structure transfers experience from seen tools to functionally similar unseen tools, directing exploration away from unrelated alternatives. ToolCompass requires no ground-truth call traces or unseen-tool access and incurs no inference overhead. Experiments on AppWorld and FTRL show consistent gains across GRPO, RFT, and DMPO. improves AppWorld OOD task success by up to 10.71 percentage points over vanilla post-training and performs best among competitive baselines on both benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 59 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

51 of 59 references.

References · 51

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  12. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
38 more
  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  3. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  4. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  5. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  6. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  7. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  8. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  9. 2024Tool Learning with Large Language Models: A Surveycited by 221
  10. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  11. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  12. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 145
  13. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agentscited by 103
  14. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
  15. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 63
  16. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 42
  17. 2025Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMscited by 28
  18. 2026TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMscited by 24
  19. 2024TOOLVERIFIER: Generalization to New Tools via Self-Verificationcited by 23
  20. 2024StepTool: Enhancing Multi-Step Tool Usage in LLMs via Step-Grained Reinforcement Learningcited by 22
  21. 2025Procedural Environment Generation for Tool-Use Agentscited by 21
  22. 2025Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environmentscited by 19
  23. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  24. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 14
  25. 2025Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactionscited by 12
  26. 2026TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agentscited by 11
  27. 2025GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulationcited by 9
  28. 2025Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Modelscited by 9
  29. 2026MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matchingcited by 9
  30. 2026ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Executioncited by 7
  31. 2026Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failurescited by 6
  32. 2026Learning from the Irrecoverable: Error-Localized Policy Optimization for Tool-Integrated LLM Reasoningcited by 5
  33. 2026Learning to Rewrite Tool Descriptions for Reliable LLM-Agent Tool Usecited by 4
  34. 2026Can RL Improve Generalization of LLM Agents? An Empirical Studycited by 4
  35. 2026PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Modelscited by 4
  36. 2025LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrievalcited by 3
  37. 2026Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Usecited by 3
  38. 2026Teaching LLMs to Learn Tool Trialing and Execution through Environment Interactioncited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction
  2. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
  3. 2026ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory
  4. 2026JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents
  5. 2026ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents
  6. 2024LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error
  7. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  8. 2026Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents