mlsanityData to 14 Oct 2026

Octopus v2: On-device language model for super agent

arXiv 2404.01744 ↗First posted 2 Apr 2024cs.CLcited by 47

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Language models have shown effectiveness in a variety of software applications, particularly in tasks related to automatic workflow. These models possess the crucial ability to call functions, which is essential in creating AI agents. Despite the high performance of large-scale language models in cloud environments, they are often associated with concerns over privacy and cost. Current on-device models for function calling face issues with latency and accuracy. Our research presents a new method that empowers an on-device model with 2 billion parameters to surpass the performance of GPT-4 in both accuracy and latency, and decrease the context length by 95%. When compared to Llama-7B with a RAG-based function calling mechanism, our method enhances latency by 35-fold. This method reduces the latency to levels deemed suitable for deployment across a variety of edge devices in production environments, aligning with the performance requisites for real-world applications.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 34 of its 63 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

34 of 63 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls6 Feb
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  3. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  4. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  5. 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  10. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  12. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
16 more
  1. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  2. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2023Active Retrieval Augmented Generationcited by 516
  5. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  6. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  7. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  8. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  9. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
  10. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  11. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  12. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  13. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  14. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  15. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agentcited by 93
  16. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 20; 2025: 16; 2026: 11.

47 papers.

Sort:

By area:

  1. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  2. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  3. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  4. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  5. 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
  6. 2024Benchmarking Mobile Device Control Agents across Diverse Configurationscited by 55
  7. 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuningcited by 44
  8. 2025Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deploymentcited by 38
  9. 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agentscited by 35
  10. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  11. 2024Foundations and Recent Trends in Multimodal Mobile Agents: A Surveycited by 31
  12. 2024Efficient and Scalable Estimation of Tool Representations in Vector Spacecited by 22
35 more
  1. 2025AutoNeural: Co-Designing Vision-Language Models for NPU Inferencecited by 17
  2. 2024Octopus v4: Graph of language modelscited by 11
  3. 2025Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deploymentcited by 10
  4. 2025AndroidGen: Building an Android Language Agent under Data Scarcitycited by 10
  5. 2024Octo-planner: On-device Language Model for Planner-Action Agentscited by 8
  6. 2024OmniVLM: A Token-Compressed, Sub-Billion-Parameter Vision-Language Model for Efficient On-Device Inferencecited by 8
  7. 2025LLM Applications: Current Paradigms and the Next Frontiercited by 8
  8. 2024Octopus v3: Technical Report for On-device Sub-billion Multimodal AI Agentcited by 7
  9. 2024Tulip Agent -- Enabling LLM-Based Agents to Solve Tasks Using Large Tool Librariescited by 7
  10. 2024Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Modelscited by 6
  11. 2024DroidCall: A Dataset for LLM-powered Android Intent Invocationcited by 5
  12. 2025Optimizing Small Language Models for In-Vehicle Function-Callingcited by 5
  13. 2025CORE: Reducing UI Exposure in Mobile Agents via Collaboration Between Cloud and Local LLMscited by 5
  14. 2025Mobile-Agent-V: A Video-Guided Approach for Effortless and Efficient Operational Knowledge Injection in Mobile Automationcited by 4
  15. 2025Towards Pervasive Distributed Agentic Generative AI -- A State of The Artcited by 4
  16. 2025ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inferencecited by 4
  17. 2026Agentic Tool Use in Large Language Modelscited by 4
  18. 2026PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Modelscited by 4
  19. 2024Elastic On-Device LLM Servicecited by 3
  20. 2024PIM-AI: A Novel Architecture for High-Efficiency LLM Inferencecited by 3
  21. 2025Mobile-Agent-V: A Video-Guided Approach for Effortless and Efficient Operational Knowledge Injection in Mobile Automationcited by 3
  22. 2024On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboardcited by 2
  23. 2024An Adaptive CSI Feedback Model Based on BiLSTM for Massive MIMO-OFDM Systemscited by 1
  24. 2024Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMscited by 1
  25. 2025PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMscited by 1
  26. 2026Beyond Scaling: Agents Are Heading to the Edgecited by 1
  27. 2026PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Languagecited by 1
  28. 2026Software Engineering for and with GUI Agentcited by 1
  29. 2026Making Prospective Memory SLM-Shaped: Typed Intention Stores for Small-Model Agentscited by 1
  30. 2024HARE: HumAn pRiors, a key to small language model Efficiencycited by 0
  31. 2026CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agentscited by 0
  32. 2026Below the Noise Floor: Bimodal Seed Collapse and Distinct Failure Modes in Small-Model Knowledge Distillationcited by 0
  33. 2026From Fixed Keys to Readable Schemas: Small Language Models for Vehicle Agent Function Callscited by 0
  34. 2026Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programscited by 0
  35. 2026SchemaWeaver: Accelerating On-Device Mobile Agentic AI via Schema-Guided Parallel LLM Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Octopus: On-device language model for function calling of software APIs
  2. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
  3. 2024TinyAgent: Function Calling at the Edge
  4. 2025OpenPhone: Mobile Agentic Foundation Models
  5. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use
  6. 2024Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
  7. 2024Understanding the Weakness of Large Language Model Agents within a Complex Android Environment
  8. 2025AgentFlux: Decoupled Fine-Tuning & Inference for On-Device Agentic Systems