mlsanityData to 14 Oct 2026

Self-Training Large Language Models for Tool-Use Without Demonstrations

arXiv 2502.05867 ↗First posted 9 Feb 2025cs.CLcited by 7

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 2 more
Abstract (arXiv)

Large language models (LLMs) remain prone to factual inaccuracies and computational errors, including hallucinations and mistakes in mathematical reasoning. Recent work augmented LLMs with tools to mitigate these shortcomings, but often requires curated gold tool-use demonstrations. In this paper, we investigate whether LLMs can learn to use tools without demonstrations. First, we analyse zero-shot prompting strategies to guide LLMs in tool utilisation. Second, we propose a self-training method to synthesise tool-use traces using the LLM itself. We compare supervised fine-tuning and preference fine-tuning techniques for fine-tuning the model on datasets constructed using existing Question Answering (QA) datasets, i.e., TriviaQA and GSM8K. Experiments show that tool-use enhances performance on a long-tail knowledge task: 3.7% on PopQA, which is used solely for evaluation, but leads to mixed results on other datasets, i.e., TriviaQA, GSM8K, and NQ-Open. Our findings highlight the potential and challenges of integrating external tools into LLMs without demonstrations.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 40 references that are in the map. First posted on arXiv 9 Feb 2025.

Builds on

32 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  2. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,916
17 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  7. 2022Transformers learn in-context by gradient descentcited by 732
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  9. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  10. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  11. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  12. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 96
  13. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 70
  14. 2024Building Math Agents with Multi-Turn Iterative Preference Learningcited by 66
  15. 2024What Are Tools Anyway? A Survey from the Language Model Perspectivecited by 54
  16. 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
  17. 2024The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Modelscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
  2. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 47
  3. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  4. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  5. 2025GRAM-R2: Self-Training Generative Foundation Reward Models for Reward Reasoningcited by 7
  6. 2025MENTOR: Reinforcement Learning via Flexible Teacher-Optimized Rewards for Tool-Use Distillationcited by 2
  7. 2026Gradually Compacting Large Language Models for Reasoning Like a Boiling Frogcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM With Tools: A Survey
  2. 2024WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models
  3. 2022Large Language Models Can Self-Improve
  4. 2025AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification
  5. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards
  6. 2024TOOLVERIFIER: Generalization to New Tools via Self-Verification
  7. 2024Self-training Large Language Models through Knowledge Detection
  8. 2024Learning to Ask: When LLM Agents Meet Unclear Instruction