mlsanityData to 7 Oct 2026

CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrieval

arXiv 2605.29271 ↗First posted 28 May 2026cs.AI cs.IR cs.LGcited by 0

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Tool retrieval over large API catalogs is a core bottleneck for LLM agents: user queries arrive in colloquial, often underspecified language, while the catalog uses technical API vocabulary that no fixed encoder can bridge on its own. The two dominant training approaches, contrastive encoder fine-tuning and HyDE-style query expansion with a frozen LLM, address this problem from opposite ends and fail in complementary directions: the fine-tuned encoder excels when the query's surface form already matches the catalog but collapses when it does not, while zero-shot HyDE is more robust to underspecified queries yet generates catalog-unaware hypothetical descriptions that degrade retrieval when queries are well-formed. We introduce CoHyDE, an iterative procedure that trains the dense encoder and the LLM rewriter as a single co-evolving system: the encoder is retrained with InfoNCE on catalog-style hypothetical descriptions produced by the rewriter, and the rewriter is preference-aligned via DPO against the encoder's retrieval scores, with both sides warm-started on the tool catalog before the loop begins. On a ~10k tool subset of the ToolBench catalog, three rounds of CoHyDE improve over the strongest single-component baseline by +2.5 pp NDCG@5 on standard queries and +6.3 pp on held-out vague queries, with gains as large as +8 pp on the hardest vague tier. Ablations confirm that co-training is the key ingredient: using either component in isolation fails to match CoHyDE on both well-formed and vague queries, with losses of up to -8 pp on vague queries.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 30 of its 39 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

30 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Tool Retrieval Bridge: Aligning Vague Instructions with Retriever Preferences via Bridge Model9 Apr
  2. 2024Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases18 Oct
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  4. 2023Gorilla: Large Language Model Connected with Massive APIs24 May

Other references · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  10. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  11. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  12. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
14 more
  1. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  2. 2019Document Expansion by Query Predictioncited by 225
  3. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  4. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  5. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  6. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  7. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 72
  8. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  9. 2024Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrievalcited by 50
  10. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 47
  11. 2022COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learningcited by 19
  12. 2024Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrievalcited by 17
  13. 2023ProTIP: Progressive Tool Retrieval Improves Planningcited by 12
  14. 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning
  2. 2026Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction
  3. 2025ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers
  4. 2026ToolWeaver: Weaving Collaborative Semantics for Scalable Tool Use in Large Language Models
  5. 2024Data-Efficient Massive Tool Retrieval: A Reinforcement Learning Approach for Query-Tool Alignment with Language Models
  6. 2026Toollery: Scaling LLM Agents to Thousands of Skills and Tools
  7. 2026Libra: Training the Environment for Agentic Information Retrieval
  8. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering