mlsanityData to 7 Oct 2026

Octopus: On-device language model for function calling of software APIs

arXiv 2404.01549 ↗First posted 2 Apr 2024cs.CL cs.SEcited by 26

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

In the rapidly evolving domain of artificial intelligence, Large Language Models (LLMs) play a crucial role due to their advanced text processing and generation abilities. This study introduces a new strategy aimed at harnessing on-device LLMs in invoking software APIs. We meticulously compile a dataset derived from software API documentation and apply fine-tuning to LLMs with capacities of 2B, 3B and 7B parameters, specifically to enhance their proficiency in software API interactions. Our approach concentrates on refining the models' grasp of API structures and syntax, significantly enhancing the accuracy of API function calls. Additionally, we propose textitconditional masking techniques to ensure outputs in the desired formats and reduce error rates while maintaining inference speeds. We also propose a novel benchmark designed to evaluate the effectiveness of LLMs in API interactions, establishing a foundation for subsequent research. Octopus, the fine-tuned model, is proved to have better performance than GPT-4 for the software APIs calling. This research aims to advance automated software development and API integration, representing substantial progress in aligning LLM capabilities with the demands of practical software engineering applications.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 36 of its 69 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

36 of 69 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls6 Feb
  2. 2023ToolQA: A Dataset for LLM Question Answering with External Tools23 Jun
  3. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  4. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
19 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  2. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  3. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  4. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  5. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 182
  6. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  7. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  8. 2023CodePlan: Repository-level Coding using LLMs and Planningcited by 139
  9. 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
  10. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
  11. 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 103
  12. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agentcited by 92
  13. 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
  14. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  15. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systemscited by 39
  16. 2023ToolCoder: Teach Code Generation Models to use API search toolscited by 37
  17. 2023The ART of LLM Refinement: Ask, Refine, and Trustcited by 29
  18. 2023Guiding Language Models of Code with Global Context using Monitorscited by 20
  19. 2023LLM As DBAcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 10; 2026: 2.

26 papers.

Sort:

By area:

  1. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  2. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  3. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  4. 2024TinyAgent: Function Calling at the Edgecited by 55
  5. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Maskingcited by 53
  6. 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Taskscited by 52
  7. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  8. 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuningcited by 44
  9. 2024LAMBDA: A Large Model Based Data Agentcited by 30
  10. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  11. 2025AutoNeural: Co-Designing Vision-Language Models for NPU Inferencecited by 17
  12. 2024MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Casescited by 13
14 more
  1. 2024Octopus v4: Graph of language modelscited by 11
  2. 2024Octopus v3: Technical Report for On-device Sub-billion Multimodal AI Agentcited by 7
  3. 2024Tulip Agent -- Enabling LLM-Based Agents to Solve Tasks Using Large Tool Librariescited by 7
  4. 2024CAMPHOR: Collaborative Agents for Multi-input Planning and High-Order Reasoning On Devicecited by 6
  5. 2024Optimizing RAG Techniques for Automotive Industry PDF Chatbots: A Case Study with Locally Deployed Ollama Modelscited by 4
  6. 2025Live API-Bench: 2500+ Live APIs for Testing Multi-Step Tool Callingcited by 4
  7. 2025Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)cited by 2
  8. 2025ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Callingcited by 2
  9. 2024Research on Tibetan Tourism Viewpoints information generation system based on LLMcited by 1
  10. 2024Less is More: Optimizing Function Calling for LLM Execution on Edge Devicescited by 1
  11. 2025Efficient On-Device Agents via Adaptive Context Managementcited by 1
  12. 2026GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Modelscited by 1
  13. 2025Intelligent IoT Attack Detection Design via ODLLM with Feature Ranking-based Knowledge Basecited by 0
  14. 2026Beyond Fluent Generation: A CPU Reliability Benchmark for MCP-Style Tool Calling in Sub-2B Small Language Models for Edge Deploymentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CallNavi, A Challenge and Empirical Study on LLM Function Calling and Routing
  2. 2024Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation
  3. 2024Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
  4. 2024Octopus v2: On-device language model for super agent
  5. 2026Beyond Perfect APIs: A Comprehensive Evaluation of LLM Agents Under Real-World API Complexity
  6. 2026LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs
  7. 2025Evaluating LLMs on Sequential API Call Through Automated Test Generation
  8. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls