Repairing Speech Access After Text Updates with Frozen Affine Fields
Not on the map. Why?
Abstract (arXiv)
Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated by text updates. Knowledge acquired through a text update can nevertheless remain difficult to access when the same query is spoken. We formulate this mismatch as post-update speech access repair, where the knowledge-bearing checkpoint stays frozen and a detachable operator repairs speech-only access in one model pass. We introduce Affine Fields for Interface Repair across Modalities (AFFIRM), which first learns and freezes a base correction for speech hidden states, then adds query-specific corrections with zero mean over the operator-training queries. Paired speech and text states provide the coordinates that condition these corrections on each query. On balanced synthetic-speech PopQA, paired comparisons do not establish a clear advantage for AFFIRM over global SALSA steering or low-rank ReFT interventions under the same frozen single-state constraint. On recorded SLUE-SQA-5 questions with text passages, a full-scale PopQA-fitted AFFIRM operator transfers without refitting and improves ROUGE-L by 0.0941 over its paired No Repair baseline. Low-rank ReFT interventions also transfer positively, whereas the matched SALSA variant falls 0.0707 below its paired baseline. These results show that text acquisition and speech access can diverge after an update and should be verified and repaired separately.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
26 of 35 references.
References · 26
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
14 more
- 2024ReFT: Representation Finetuning for Language Modelscited by 197
- 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
- 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 35
- 2025Closing the Gap Between Text and Speech Understanding in LLMscited by 17
- 2026X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMscited by 16
- 2025Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Modelscited by 13
- 2026CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillationcited by 11
- 2023GSQA: An End-to-End Model for Generative Spoken Question Answeringcited by 9
- 2026Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gapcited by 7
- 2026SALSA: Speech Aware LLM Adaptation via Learned Steering Activation Vectorscited by 4
- 2025Steer-MoE: Efficient Audio-Language Alignment with a Mixture-of-Experts Steering Modulecited by 3
- 2026Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?cited by 1
- 2026Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Interventioncited by 1
- 2026Is Text All You Need? Text as a Universal Information Bottleneck for Speech LLMscited by 1
Built on by
Not cited yet.