# Adaptive Edge-Cloud Inference for Speech-to-Action Systems Using ASR and Large Language Models

arXiv: https://arxiv.org/abs/2512.12769 · HTML: https://mlsanity.com/paper/2512.12769/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 14 Dec 2025.

## Builds on (most cited first, 18 of 18)

- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [On decoder-only architecture for speech-to-text and large language model integration](https://mlsanity.com/paper/2307.03917/) (2023)
- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)
- [Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization](https://mlsanity.com/paper/2404.09956/) (2024)
- [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://mlsanity.com/paper/1710.07654/) (2017)
- [Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition](https://mlsanity.com/paper/2407.04675/) (2024)
- [SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model](https://mlsanity.com/paper/2104.05557/) (2021)
- [LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM](https://mlsanity.com/paper/2503.04724/) (2025)
- [Sanitizing Manufacturing Dataset Labels Using Vision-Language Models](https://mlsanity.com/paper/2506.23465/) (2025)
- [Transforming LLMs into Cross-modal and Cross-lingual Retrieval Systems](https://mlsanity.com/paper/2404.01616/) (2024)
- [Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models](https://mlsanity.com/paper/2507.07877/) (2025)
- [Efficient Routing of Inference Requests across LLM Instances in Cloud-Edge Computing](https://mlsanity.com/paper/2507.15553/) (2025)
- [AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents](https://mlsanity.com/paper/2510.00039/) (2025)
- [Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge](https://mlsanity.com/paper/2411.13766/) (2024)
- [Taxonomic Classification of IoT Smart Home Voice Control](https://mlsanity.com/paper/2210.15656/) (2022)
- [HySim-LLM: Embedding-Weighted Fine-Tuning Bounds and Manifold Denoising for Domain-Adapted LLMs](https://mlsanity.com/paper/2510.07796/) (2025)
- [Bayes-Split-Edge: Bayesian Optimization for Constrained Collaborative Inference in Wireless Edge Systems](https://mlsanity.com/paper/2510.23503/) (2025)

## Built on by (most cited first, 1 of 1)

- [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026)

## Abstract (arXiv)

Voice-based interaction has emerged as a natural and intuitive modality for controlling IoT devices. However, speech-driven edge devices face a fundamental trade-off between cloud-based solutions, which offer stronger language understanding capabilities at the cost of latency, connectivity dependence, and privacy concerns, and edge-based solutions, which provide low latency and improved privacy but are limited by computational constraints. This paper presents ASTA, an adaptive speech-to-action solution that dynamically routes voice commands between edge and cloud inference to balance performance and system resource utilization. ASTA integrates on-device automatic speech recognition and lightweight offline language-model inference with cloud-based LLM processing, guided by real-time system metrics such as CPU workload, device temperature, and network latency. A metric-aware routing mechanism selects the inference path at runtime, while a rule-based command validation and repair component ensures successful end-to-end command execution. We implemented our solution on an NVIDIA Jetson-based edge platform and evaluated it using a diverse dataset of 80 spoken commands. Experimental results show that ASTA successfully routes all input commands for execution, achieving a balanced distribution between online and offline inference. The system attains an ASR accuracy of 62.5% and generates executable commands without repair for only 47.5% of inputs, highlighting the importance of the repair mechanism in improving robustness. These results suggest that adaptive edge-cloud orchestration is a viable approach for resilient and resource-aware voice-controlled IoT systems.