mlsanityData to 14 Oct 2026

Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

arXiv 2608.05126 ↗First posted 5 Aug 2026cs.CL cs.MMcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 19 of its 45 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

19 of 45 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  10. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  12. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
7 more
  1. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  2. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  3. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  4. 2024ToolACE: Winning the Points of LLM Function Callingcited by 216
  5. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
  6. 2025Defeating the Training-Inference Mismatch via FP16cited by 66
  7. 2020A Co-Interactive Transformer for Joint Slot Filling and Intent Detectioncited by 61

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark
  2. 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Models
  3. 2026Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Models
  4. 2025VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models
  5. 2025URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models
  6. 2025Spoken Conversational Agents with Large Language Models
  7. 2024Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
  8. 2023SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents