mlsanityData to 14 Oct 2026

LLMs are General Asynchronous Agents

arXiv 2609.35427 ↗First posted 28 Sep 2026cs.LG cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks also fits: Agent memory systems

Abstract (arXiv)

Modern LLMs are increasingly capable as autonomous agents, but they follow sequential interaction cycles: read, think, reply or call tools, repeat. Many real-world use cases are not sequential: voice assistants, embodied agents, and monitoring systems receive new inputs while they think or perform another task. Modern LLMs address this with specialized architectures for voice interaction and video streams, VLAs for robot control, asynchronous tool calling for API usage, and others. In this work, we generalize from different asynchronous tasks to general asynchronous agents that can adapt to different types of concurrency. To achieve this, we develop an asynchronous LLM framework that lets users (or the agents themselves) define inference coroutines with overlapping memory states. We showcase that Qwen 3.x models are capable of asynchronous operation for streaming video understanding, videogames, and monitoring, without task-specific training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 151 of its 196 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

151 of 196 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AsyncVoice Agent: Real-Time Explanation for LLM Planning and Reasoning17 Oct
  2. 2025Robotouille: An Asynchronous Planning Benchmark for LLM Agents6 Feb
  3. 2024Asynchronous Tool Usage for Real-Time Agents28 Oct
  4. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents23 Sep
  5. 2024ReDel: A Toolkit for LLM-Powered Recursive Multi-Agent Systems5 Aug

Other references · 146

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  10. 2020Longformer: The Long-Document Transformercited by 3,413
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  12. 2023Mistral 7Bcited by 3,164
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  13. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  14. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  15. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  16. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  17. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  18. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  19. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  20. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  21. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  22. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  23. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  24. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  25. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  26. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  27. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  28. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  29. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  30. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  31. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  32. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  33. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  34. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  35. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  36. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  37. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  38. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529

50 most cited of 146 All 146 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction
  2. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  3. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
  4. 2025Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games
  5. 2025A Concurrent Modular Agent: Framework for Autonomous LLM Agents
  6. 2026AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
  7. 2023LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination
  8. 2026TypeGo: An OS Runtime for Embodied Agents