Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored. We analyze the predictive behavior encoded in FD-SLM hidden representations and find that they exhibit stream-specific predictive patterns: during listening, they preferentially predict the incoming user stream, whereas during speaking, they preferentially predict the model output stream. Building on this observation, we show that FD-SLMs dynamically modulate their internal predictive focus between two states: a generative state aligned with model output generation and a perceptive state aligned with incoming user input. However, this modulation can lag behind abrupt changes in conversational context. During user interruptions, the model remains transiently biased toward the generative state before transitioning into the perceptive state, causing it to miss the beginning of the incoming input. We term this delayed internal transition state inertia. To quantify its downstream impact, we introduce the Zero-Buffer Benchmark (ZBB), a diagnostic benchmark for evaluating immediate interruption comprehension when user speech begins abruptly. We evaluate this setting using response correctness and initial-word occurrence rate (IWOR). Finally, we mitigate state inertia through activation steering with a perception vector, a training-free intervention with little additional computational overhead. Across multiple state-of-the-art FD-SLMs, activation steering substantially improves interruption handling; for example, on PersonaPlex, it improves correctness from 28% to 45% and IWOR from 40% to 72% without any fine-tuning.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 38 of its 51 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
38 of 51 references.
Closest ancestors · 3
Same direction, newest first
- 2024Improving Instruction-Following in Language Models through Activation Steering15 Oct
- 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
- 2023Steering Language Models With Activation Engineering20 Aug
Other references · 35
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
- 2022High Fidelity Neural Audio Compressioncited by 687
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2025Qwen3-Omni Technical Reportcited by 497
- 2023Simple and Controllable Music Generationcited by 488
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
23 more
- 2025Kimi-Audio Technical Reportcited by 222
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
- 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
- 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
- 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
- 2025Step-Audio 2 Technical Reportcited by 105
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
- 2024WavChat: A Survey of Spoken Dialogue Modelscited by 90
- 2024Recent Advances in Speech Language Models: A Surveycited by 87
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 72
- 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 63
- 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 54
- 2025Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examinercited by 41
- 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
- 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 30
- 2024Codec-SUPERB: An In-Depth Analysis of Sound Codec Modelscited by 26
- 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
- 2025MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Modelscited by 22
- 2025FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systemscited by 20
- 2025Game-Time: Evaluating Temporal Dynamics in Spoken Language Modelscited by 13
- 2026Full-Duplex Interaction in Spoken Dialogue Systems: A Comprehensive Study from the ICASSP 2026 HumDial Challengecited by 13
- 2026Semantic-Aware Interruption Detection in Spoken Dialogue Systems: Benchmark, Metric, and Modelcited by 4
- 2023Voice Activity Detection (VAD) in Noisy Environmentscited by 2
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Latent-IM: Latent Interaction Management for Speech LLMscited by 0
- 2026Controlling Speaking Rate in Autoregressive TTS via Activation Steeringcited by 0
- 2026Coupled but Late: Turn-Taking Between Full-Duplex Speech Models in Unscripted Dialoguecited by 0
- 2026Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Modelcited by 0
- 2026Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Model6 Oct 2026
- 2026Coupled but Late: Turn-Taking Between Full-Duplex Speech Models in Unscripted Dialogue6 Oct 2026
- 2026Controlling Speaking Rate in Autoregressive TTS via Activation Steering27 Sep 2026
- 2026Latent-IM: Latent Interaction Management for Speech LLMs29 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion
- 2026RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
- 2026Full-duplex Spoken Language Modeling with Discrete Diffusion
- 2026SteerDuplex: Steerable Duplex Speech Dialogue Models
- 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
- 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models
- 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Models
- 2026Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents