mlsanityData to 14 Oct 2026

From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents

arXiv 2610.15390 ↗First posted 11 Oct 2026cs.CR cs.CV cs.LGcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Audio Large Language Models (Audio LLMs) are embedded in agents for smart homes, vehicles, wearables, and embodied systems, where acoustic inputs can propagate from perception into persistent state and consequential execution. This setting extends security analysis beyond model behavior to the system conditions that determine how acoustic inputs can influence agent behavior and external actions. Existing surveys examine acoustic security, LLM-based agents, and edge deployment in isolation, leaving their security interactions in deployed Audio LLM-based agents insufficiently characterized. This survey introduces an Acoustic Sensing-to-Action (S2A) system model that traces these security implications along the path from acoustic sensing to external execution, while accounting for edge-cloud runtime and lifecycle transformation. We use the model to synthesize attacks, defenses, and evaluation methods across the acoustic, agent, and edge-system literature. The synthesis shows that security gaps can emerge within components and at the transitions between stages, where protections developed for one stage do not necessarily provide end-to-end assurance in the deployed system. These findings motivate security evaluation that follows acoustic influence from sensing to action across persistent state, system boundaries, and changing deployment conditions in real-world agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 143 of its 179 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

143 of 179 references.

Closest ancestors · 31

Same direction, newest first

  1. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systems24 Aug
  2. 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happened17 Aug
  3. 2026Memory Provenance Laundering in LLM Agents: A Non-Amplification Firewall for Persistent Memory31 Jul
  4. 2026Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses6 Jul
  5. 2026LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity26 Jun
  6. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun
  7. 2026From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents3 Jun
  8. 2026Evo-Attacker: Memory-Augmented Reinforcement Learning for Long-Horizon Tool Attacks on LLM-MAS25 May
  9. 2026Hidden in Memory: Sleeper Memory Poisoning in LLM Agents14 May
  10. 2026AgenTEE: Confidential LLM Agent Execution on Edge Devices20 Apr
  11. 2026Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoning18 Apr
  12. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle17 Apr
19 more
  1. 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use7 Apr
  2. 2026Aegis: Towards Governance, Integrity, and Security of AI Voice Agents7 Feb
  3. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
  4. 2026Towards Verifiably Safe Tool Use for LLM Agents12 Jan
  5. 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit9 Jan
  6. 2026BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents8 Jan
  7. 2025AegisMCP: Online Graph Intrusion Detection for Tool-Augmented LLMs on Edge Devices22 Oct
  8. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory29 Sep
  9. 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
  10. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  11. 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents4 Apr
  12. 2025Defeating Prompt Injections by Design24 Mar
  13. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
  14. 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction5 Mar
  15. 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
  16. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
  17. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  18. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  19. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan

Other references · 112

  1. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  2. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  3. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  4. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  5. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  6. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  7. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 274
  8. 2021ASVspoof 2021: accelerating progress in spoofed and deepfake speech detectioncited by 161
  9. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  10. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 124
  11. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111
  12. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
38 more
  1. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 82
  2. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
  3. 2017DolphinAtack: Inaudible Voice Commandscited by 69
  4. 2018CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognitioncited by 66
  5. 2020DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environmentscited by 58
  6. 2023LLaSM: Large Language and Speech Modelcited by 58
  7. 2020DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flipscited by 56
  8. 2018Robust Audio Adversarial Example for a Physical Attackcited by 44
  9. 2024Voice Jailbreak Attacks Against GPT-4ocited by 42
  10. 2024The VoicePrivacy 2024 Challenge Evaluation Plancited by 41
  11. 2021WaveGuard: Understanding and Mitigating Audio Adversarial Examplescited by 40
  12. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  13. 2024AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Modelscited by 38
  14. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbationcited by 37
  15. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 35
  16. 2025Safety Guardrails for LLM-Enabled Robotscited by 35
  17. 2024NatureLM-audio: an Audio-Language Foundation Model for Bioacousticscited by 33
  18. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMscited by 31
  19. 2025URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Modelscited by 30
  20. 2021Quantization Backdoors to Deep Learning Commercial Frameworkscited by 29
  21. 2024PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuningcited by 26
  22. 2024LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystemcited by 24
  23. 2023Defending against Adversarial Audio via Diffusion Modelcited by 23
  24. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 23
  25. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 22
  26. 2020ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networkscited by 21
  27. 2022Going In Style: Audio Backdoors Through Stylistic Transformationscited by 21
  28. 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Modelscited by 21
  29. 2026PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?cited by 21
  30. 2020Light Commands: Laser-Based Audio Injection Attacks on Voice-Controllable Systemscited by 18
  31. 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 18
  32. 2025Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasetscited by 18
  33. 2025AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Modelscited by 18
  34. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 18
  35. 2025VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Modelscited by 17
  36. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 17
  37. 2025JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Modelscited by 16
  38. 2025SimuHome: A Temporal- and Environment-Aware Benchmark for Smart Home LLM Agentscited by 15

50 most cited of 112 All 112 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Piggybacking on Perception: Stealthy Concurrent Audio Prompt Injections against Multimodal LLM Agents
  2. 2026From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systems
  3. 2026Security Engineering of OpenClaw: Analyzing Attack Surface Expansion and Trust-Boundary Violations
  4. 2025Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks
  5. 2026Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats
  6. 2026Semantic Denial of Service in LLM-controlled robots
  7. 2025Trust in LLM-controlled Robotics: a Survey of Security Threats, Defenses and Challenges
  8. 2025AegisAgent: An Autonomous Defense Agent Against Prompt Injection Attacks in LLM-HARs