mlsanityData to 14 Oct 2026

Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering

arXiv 2606.11386 ↗First posted 9 Jun 2026cs.CL cs.AI eess.AScited by 4

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored. We analyze the predictive behavior encoded in FD-SLM hidden representations and find that they exhibit stream-specific predictive patterns: during listening, they preferentially predict the incoming user stream, whereas during speaking, they preferentially predict the model output stream. Building on this observation, we show that FD-SLMs dynamically modulate their internal predictive focus between two states: a generative state aligned with model output generation and a perceptive state aligned with incoming user input. However, this modulation can lag behind abrupt changes in conversational context. During user interruptions, the model remains transiently biased toward the generative state before transitioning into the perceptive state, causing it to miss the beginning of the incoming input. We term this delayed internal transition state inertia. To quantify its downstream impact, we introduce the Zero-Buffer Benchmark (ZBB), a diagnostic benchmark for evaluating immediate interruption comprehension when user speech begins abruptly. We evaluate this setting using response correctness and initial-word occurrence rate (IWOR). Finally, we mitigate state inertia through activation steering with a perception vector, a training-free intervention with little additional computational overhead. Across multiple state-of-the-art FD-SLMs, activation steering substantially improves interruption handling; for example, on PersonaPlex, it improves correctness from 28% to 45% and IWOR from 40% to 72% without any fine-tuning.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 38 of its 51 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

38 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Improving Instruction-Following in Language Models through Activation Steering15 Oct
  2. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  3. 2023Steering Language Models With Activation Engineering20 Aug

Other references · 35

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  2. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  5. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  6. 2022High Fidelity Neural Audio Compressioncited by 687
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  8. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  9. 2025Qwen3-Omni Technical Reportcited by 497
  10. 2023Simple and Controllable Music Generationcited by 488
  11. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  12. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
23 more
  1. 2025Kimi-Audio Technical Reportcited by 222
  2. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  3. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
  4. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  5. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  6. 2025Step-Audio 2 Technical Reportcited by 105
  7. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  8. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 90
  9. 2024Recent Advances in Speech Language Models: A Surveycited by 87
  10. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 72
  11. 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 63
  12. 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 54
  13. 2025Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examinercited by 41
  14. 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
  15. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 30
  16. 2024Codec-SUPERB: An In-Depth Analysis of Sound Codec Modelscited by 26
  17. 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
  18. 2025MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Modelscited by 22
  19. 2025FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systemscited by 20
  20. 2025Game-Time: Evaluating Temporal Dynamics in Spoken Language Modelscited by 13
  21. 2026Full-Duplex Interaction in Spoken Dialogue Systems: A Comprehensive Study from the ICASSP 2026 HumDial Challengecited by 13
  22. 2026Semantic-Aware Interruption Detection in Spoken Dialogue Systems: Benchmark, Metric, and Modelcited by 4
  23. 2023Voice Activity Detection (VAD) in Noisy Environmentscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion
  2. 2026RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
  3. 2026Full-duplex Spoken Language Modeling with Discrete Diffusion
  4. 2026SteerDuplex: Steerable Duplex Speech Dialogue Models
  5. 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
  6. 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models
  7. 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Models
  8. 2026Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents