Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models
On the map: Speech and audio language models → Audio language models and spoken dialogue
Abstract (arXiv)
Full-duplex spoken language models decide at every frame whether to take the floor, defining a conversational personality -- yet each set of weights encodes exactly one, which human speakers bracket rather than share. Existing controls mostly act from outside; none identifies the quantity that sets how much the model speaks. We identify it: a single late-layer residual-stream direction, the initiative vector, extracted from the model's own conversations without training or annotation. The same direction reads the decision, anticipating onsets up to 800 ms ahead, and shifts it: scaled during generation, it moves speech ratio from 6-12% to 42-61% (no steering: 21-30%), consistently across conversations, weights, and corpora. A short reference recording suffices to transfer a speaker's turn-taking style to the model, and the transfer is reference-specific: swapping in another speaker's reference worsens the match in 93-98% of cases, whereas the best uniform dose is indistinguishable from no steering. Turn-taking style is not frozen into the weights but a direction that can be read, shifted, and calibrated per person. Audio examples are available at https://elpsykongloo.github.io/initiative-vector/.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Audio language models and spoken dialogue, one of 11 directions in Speech and audio language models. Builds on 24 of its 28 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
24 of 28 references.
Closest ancestors · 12
Same direction, newest first
- 2026SteerDuplex: Steerable Duplex Speech Dialogue Models11 Sep
- 2026Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning8 Jul
- 2026Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering9 Jun
- 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models9 Jun
- 2026Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Models19 May
- 2026Instruct-FD: Can Your Full-Duplex Speech System Follow Turn-Taking Instructions?15 May
- 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models14 Jan
- 2025From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Models18 Sep
- 2025Prompt-Guided Turn-Taking Prediction26 Jun
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents23 Sep
- 2024Moshi: a speech-text foundation model for real-time dialogue17 Sep
- 2022Generative Spoken Dialogue Language Modeling30 Mar
Other references · 12
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,131
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,065
- 2023Steering Language Models With Activation Engineeringcited by 993
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 745
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 371
- 2022Voice Activity Projection: Self-supervised Learning of Turn-taking Eventscited by 66
- 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 58
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 46
- 2023A Survey of Personality, Persona, and Profile in Conversational Agents and Chatbotscited by 6
- 2026F-Actor: Controllable Conversational Behaviour in Full-Duplex Modelscited by 5
- 2025"Dyadosyncrasy", Idiosyncrasy and Demographic Factors in Turn-Takingcited by 3
- 2026Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Frameworkcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue
- 2025Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics
- 2024Large Language Models Know What To Say But Not When To Speak
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
- 2026Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models
- 2026DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining
- 2023Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis
- 2025Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems