mlsanityData to 7 Oct 2026

Asynchronous LLM Function Calling

arXiv 2412.07017 ↗First posted 9 Dec 2024cs.CL cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) use function calls to interface with external tools and data source. However, the current approach to LLM function calling is inherently synchronous, where each call blocks LLM inference, limiting LLM operation and concurrent function execution. In this work, we propose AsyncLM, a system for asynchronous LLM function calling. AsyncLM improves LLM's operational efficiency by enabling LLMs to generate and execute function calls concurrently. Instead of waiting for each call's completion, AsyncLM introduces an interrupt mechanism to asynchronously notify the LLM in-flight when function calls return. We design an in-context protocol for function calls and interrupts, provide fine-tuning strategy to adapt LLMs to the interrupt semantics, and implement these mechanisms efficiently on LLM inference process. We demonstrate that AsyncLM can reduce end-to-end task completion latency from 1.6x-5.4x compared to synchronous function calling on a set of benchmark tasks in the Berkeley function calling leaderboard (BFCL). Furthermore, we discuss how interrupt mechanisms can be extended to enable novel human-LLM or LLM-LLM interactions.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 13 of its 37 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

13 of 37 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  4. 2024Understanding the planning of LLM agents: A surveycited by 414
  5. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  6. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  7. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  8. 2023Stateful Large Language Model Serving with Pensievecited by 59
  9. 2024Fast State Restoration in LLM Serving with HCachecited by 32
  10. 2023TypeFly: Flying Drones with Large Language Modelcited by 23
  11. 2024Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Executioncited by 16
  12. 2024An LLM-Tool Compiler for Fused Parallel Function Callingcited by 14
1 more
  1. 2024LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Cachingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 8.

18 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 43
  3. 2026Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflowscited by 27
  4. 2025Accelerating Earth Science Discovery via Multi-Agent LLM Systemscited by 13
  5. 2025SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflowscited by 12
  6. 2025A Survey on Parallel Reasoningcited by 11
  7. 2025Asynchronous Reasoning: Training-Free Interactive Thinking LLMscited by 6
  8. 2026IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agentscited by 4
  9. 2025ATTS: Asynchronous Test-Time Scaling via Conformal Predictioncited by 3
  10. 2026Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Callingcited by 2
  11. 2026Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMscited by 2
  12. 2026SimpleTool: Parallel Decoding for Real-Time LLM Function Callingcited by 1
6 more
  1. 2026Never Stop Thinking: Continuous-Time Language Agentscited by 1
  2. 2025ODIA: Oriented Distillation for Inline Acceleration of LLM-based Function Callingcited by 0
  3. 2025From Specification to Service: Accelerating API-First Development Using Multi-Agent Systemscited by 0
  4. 2025Serve Programs, Not Promptscited by 0
  5. 2026AI Agent for Reverse-Engineering Legacy Finite-Difference Code and Translating to Devitocited by 0
  6. 2026Second Thought: Reasoning in Parallel as LLM Agents Act and Observecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023An LLM Compiler for Parallel Function Calling
  2. 2025Efficient Function Orchestration for Large Language Models
  3. 2024Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning
  4. 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling
  5. 2024Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
  6. 2024PIMphony: Overcoming Bandwidth and Capacity Inefficiency in PIM-based Long-Context LLM Inference System
  7. 2024Llumnix: Dynamic Scheduling for Large Language Model Serving
  8. 2025Digging Into the Internal: Causality-Based Analysis of LLM Function Calling