mlsanityData to 19 Oct 2026

Casting Everything to Online API Services? A Survey of Integrating Localized Speech Recognition Models in Robotic Systems

arXiv 2607.11792 ↗First posted 13 Jul 2026cs.RO eess.AScited by 0

On the map: Speech and audio language models → ASR evaluation, bias and atypical speech

Abstract (arXiv)

Automatic speech recognition (ASR) has become a critical component of modern robotic systems because it is one of the most natural and intuitive ways for humans to interact with robots. A commonly used method is to directly use API services online. But is that all we can do? This article provides an overview of how ASR technologies are integrated into various intelligent robots and machines. We discuss the evolution of speech recognition from established approaches to state-of-the-art deep learning models, such as OpenAI's Whisper. We also list large-scale datasets and open source toolkits that have been widely used in both industry and academia. We structure the survey around ASR model families, deployment strategies in robotics (especially ROS-based, cloud-based, and hybrid solutions), and several real-world robotic platforms. Finally, we outline the challenges of deploying robust speech recognition in robots and discuss future directions, including multimodal interaction in diverse and dynamic environments. This paper can help social robotics researchers better navigate the emerging domain of language-based natural human-robot interaction.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in ASR evaluation, bias and atypical speech, one of 11 directions in Speech and audio language models. Builds on 28 of its 61 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

28 of 61 references.

References · 28

  1. 2017Attention Is All You Needcited by 61,180
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,923
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,227
  6. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,596
  7. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,311
  8. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,218
  9. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 754
  10. 2015Listen, Attend and Spellcited by 635
  11. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 532
  12. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 497
16 more
  1. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 495
  2. 2023Scaling Speech Technology to 1,000+ Languagescited by 483
  3. 2015Attention-Based Models for Speech Recognitioncited by 480
  4. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 270
  5. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 95
  6. 2020Spelling Error Correction with Soft-Masked BERTcited by 92
  7. 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 85
  8. 2020Curriculum Pre-training for End-to-End Speech Translationcited by 58
  9. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 57
  10. 2020Distilling the Knowledge of BERT for Sequence-to-Sequence ASRcited by 45
  11. 2024Understanding Large-Language Model (LLM)-powered Human-Robot Interactioncited by 45
  12. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 41
  13. 2018Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languagescited by 28
  14. 2018Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinesecited by 26
  15. 2018A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinesecited by 16
  16. 2025Applying General Turn-taking Models to Conversational Human-Robot Interactioncited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Hybrid ASR for Resource-Constrained Robots: HMM - Deep Learning Fusion
  2. 2025Talking to Robots: A Practical Examination of Speech Foundation Models for HRI Applications
  3. 2024Child Speech Recognition in Human-Robot Interaction: Problem Solved?
  4. 2022Can Visual Context Improve Automatic Speech Recognition for an Embodied Agent?
  5. 2026Back to Basics: Revisiting ASR in the Age of Voice Agents
  6. 2024Multimodal Human-Autonomous Agents Interaction Using Pre-Trained Language and Visual Foundation Models
  7. 2021Romanian Speech Recognition Experiments from the ROBIN Project
  8. 2023Automatic Speech Recognition of Non-Native Child Speech for Language Learning Applications