Localizing and Editing Knowledge in Large Audio-Language Models
Not on the map. Why?
Abstract (arXiv)
Large Audio-Language Models (LALMs) have shown strong performance in speech understanding, making speech a natural interface for accessing factual information. Yet they are trained on static corpora and may encode incorrect facts. Existing model editing methods localize and update facts in text-only LLMs, but do not account for continuous speech representations, or where knowledge is stored across acoustic or language modules, or their cross-modal module. We construct the first audio benchmark for knowledge localization and editing in LALMs and propose a speech-driven locate-then-edit framework. First, we use speech-aware causal tracing to localize layers and modules that support factual retrieval and then apply editing at identified sites. Experiments show that factual knowledge is jointly encoded in audio and text modules, and that audio editing yields more effective updates than text editing or fine-tuning, enabling fine-grained knowledge control in speech AI systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 22 references that are in the map. First posted on arXiv 15 Mar 2026.
Builds on
20 of 22 references.
References · 20
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2022Mass-Editing Memory in a Transformercited by 976
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2024Qwen2-Audio Technical Reportcited by 543
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
- 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 320
- 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
- 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
- 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 151
8 more
- 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
- 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 75
- 2024SpeechVerse: A Large-scale Generalizable Audio Language Modelcited by 62
- 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 48
- 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 48
- 2024Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Modelscited by 30
- 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 29
- 2022token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Textcited by 3
Built on by
Not cited yet.