mlsanityData to 14 Oct 2026

Hammer: Robust Function-Calling for On-Device Language Models via Function Masking

arXiv 2410.04587 ↗First posted 6 Oct 2024cs.LG cs.AI cs.SEcited by 55

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Large language models have demonstrated impressive value in performing as autonomous agents when equipped with external tools and API calls. Nonetheless, effectively harnessing their potential for executing complex tasks crucially relies on enhancements in their function calling capabilities. This paper identifies a critical gap in existing function calling models, where performance varies significantly across benchmarks, often due to being misled by specific naming conventions. To address such an issue, we introduce Hammer, a novel family of foundation models specifically engineered for on-device function calling. Hammer employs an augmented dataset that enhances models' sensitivity to irrelevant functions and incorporates function masking techniques to minimize misleading. Our empirical evaluations reveal that Hammer not only outperforms larger models but also demonstrates robust generalization across diverse benchmarks, achieving sota results. Our open source contributions include a specialized dataset for irrelevance detection, a tuning framework for enhanced generalization, and the Hammer models, establishing a new standard for function calling performance.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 19 of its 27 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

19 of 27 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024TinyAgent: Function Calling at the Edge1 Sep
  2. 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks27 Jun
  3. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
  4. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark14 May
  5. 2024Octopus: On-device language model for function calling of software APIs2 Apr
  6. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs23 Feb
  7. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  8. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr

Other references · 11

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  7. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  9. 2023An LLM Compiler for Parallel Function Callingcited by 179
  10. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 111
  11. 2024Reinforcing Language Agents via Policy Optimization with Action Decompositioncited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 25; 2026: 26.

55 papers.

Sort:

By area:

  1. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  2. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  4. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 71
  5. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  6. 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 32
  7. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 24
  8. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  9. 2024Agentic Information Retrievalcited by 21
  10. 2025Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Modelcited by 20
  11. 2025Tool Preferences in Agentic LLMs are Unreliablecited by 18
  12. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Modelscited by 17
38 more
  1. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  2. 2026Benchmarking LLM Tool-Use in the Wildcited by 17
  3. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  4. 2025Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missionscited by 12
  5. 2025Automating High Energy Physics Data Analysis with LLM-Powered Agentscited by 10
  6. 2025ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learningcited by 9
  7. 2025TTPA: Token-level Tool-use Preference Alignment Training Framework with Fine-grained Evaluationcited by 9
  8. 2025iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Usecited by 7
  9. 2025ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolutioncited by 7
  10. 2025Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklistscited by 7
  11. 2025C3-Bench: The Things Real Disturbing LLM based Agent in Multi-Taskingcited by 4
  12. 2025ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"cited by 4
  13. 2026Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectoriescited by 4
  14. 2025MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Modelscited by 3
  15. 2025LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrievalcited by 3
  16. 2025ToolSample: Dual Dynamic Sampling Methods with Curriculum Learning for RL-based Tool Learningcited by 3
  17. 2025AgentFlux: Decoupled Fine-Tuning & Inference for On-Device Agentic Systemscited by 3
  18. 2026DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learningcited by 3
  19. 2026GD2PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimizationcited by 3
  20. 2025A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Designcited by 2
  21. 2026AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraintscited by 2
  22. 2026Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Callingcited by 2
  23. 2026Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitivescited by 2
  24. 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
  25. 2024HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Device Scenarioscited by 1
  26. 2025Declarative Techniques for NL Queries over Heterogeneous Datacited by 1
  27. 2025Efficient On-Device Agents via Adaptive Context Managementcited by 1
  28. 2026Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agentscited by 1
  29. 2026SGA-MCTS: Decoupling Planning from Execution via Training-Free Atomic Experience Retrievalcited by 1
  30. 2026D-CORE: Incentivizing Task Decomposition in Large Reasoning Models for Complex Tool Usecited by 0
  31. 2026From Business Events to Auditable Decisions: Ontology-Governed Graph Simulation for Enterprise AIcited by 0
  32. 2026Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Studycited by 0
  33. 2026Attributing Structured-Output Gains in Function Calling: Interface Alignment versus Procedural Transfercited by 0
  34. 2026DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Serverscited by 0
  35. 2026Data Turnstile: A Scalable Open Framework for Function-Calling Data Generationcited by 0
  36. 2026UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMscited by 0
  37. 2026AgentWeave: Routing Before Reasoning for Efficient Function Calling in Tool-Rich Language Modelscited by 0
  38. 2026Invocation-Level Reliability of Tool-Using Agentscited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Robustness of Agentic Function Calling
  2. 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling
  3. 2026Small Reasoning Models are Instruction Followers in Function Calling
  4. 2024Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
  5. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls
  6. 2024ToolACE: Winning the Points of LLM Function Calling
  7. 2024Benchmarking Floworks against OpenAI & Anthropic: A Novel Framework for Enhanced LLM Function Calling
  8. 2026The "I Don't Know" Filter: Enhancing Agentic Reliability in Function Calling