mlsanityData to 7 Oct 2026

Asynchronous Tool Usage for Real-Time Agents

arXiv 2410.21620 ↗First posted 28 Oct 2024cs.AIcited by 13

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While frontier large language models (LLMs) are capable tool-using agents, current AI systems still operate in a strict turn-based fashion, oblivious to passage of time. This synchronous design forces user queries and tool-use to occur sequentially, preventing the systems from multitasking and reducing interactivity. To address this limitation, we introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting, integrated with automatic speech recognition and text-to-speech. Drawing inspiration from the concepts originally developed for real-time operating systems, this work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 9 of its 35 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

9 of 35 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration23 May
  2. 2024Embodied LLM Agents Learn to Cooperate in Organized Teams19 Mar
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  4. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  5. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents5 Jun

Other references · 4

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  3. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  4. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 6.

13 papers.

Sort:

By area:

  1. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 43
  2. 2025Large Language Models Miss the Multi-Agent Markcited by 25
  3. 2025Real-Time Reasoning Agents in Evolving Environmentscited by 13
  4. 2025Evaluating Speech-to-Text x LLM x Text-to-Speech Combinations for AI Interview Systemscited by 3
  5. 2025ATTS: Asynchronous Test-Time Scaling via Conformal Predictioncited by 3
  6. 2026AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarioscited by 3
  7. 2025AsyncVoice Agent: Real-Time Explanation for LLM Planning and Reasoningcited by 2
  8. 2026Engagement Process: Rethinking the Temporal Interface of Action and Observationcited by 2
  9. 2026Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMscited by 2
  10. 2026DuCCAE: A Hybrid Engine for Immersive Conversation via Collaboration, Augmentation, and Evolutioncited by 1
  11. 2025A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistantscited by 0
  12. 2026Continuous Interaction Diffusion: A Diffusion-Native Runtime for Asynchronous Tool-Augmented Reasoningcited by 0
1 more
  1. 2026LLMs are General Asynchronous Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
  2. 2025MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations
  3. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  4. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage
  5. 2025Task Memory Engine (TME): Enhancing State Awareness for Multi-Step LLM Agent Tasks
  6. 2024Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset
  7. 2025DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systems
  8. 2026Long-term Task-oriented Agent: Proactive Long-term Intent Maintenance in Dynamic Environments