# From Independence to Interaction: Speaker-Aware Simulation of Multi-Speaker Conversational Timing

arXiv: https://arxiv.org/abs/2509.15808 · HTML: https://mlsanity.com/paper/2509.15808/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [Speaker-Aware Simulation Improves Conversational Speech Recognition](https://mlsanity.com/paper/2602.04776/) (2026)
- [LibriConvo: Simulating Conversations from Read Literature for ASR and Diarization](https://mlsanity.com/paper/2510.23320/) (2025)
- [Efficient ASR Training with Conversations that Never Happened](https://mlsanity.com/paper/2606.03957/) (2026)

## Abstract (arXiv)

We present a speaker-aware approach for simulating multi-speaker conversations that captures temporal consistency and realistic turn-taking dynamics. Prior work typically models aggregate conversational statistics under an independence assumption across speakers and turns. In contrast, our method uses speaker-specific deviation distributions enforcing intra-speaker temporal consistency, while a Markov chain governs turn-taking and a fixed room impulse response preserves spatial realism. We also unify pauses and overlaps into a single gap distribution, modeled with kernel density estimation for smooth continuity. Evaluation on Switchboard using intrinsic metrics - global gap statistics, correlations between consecutive gaps, copula-based higher-order dependencies, turn-taking entropy, and gap survival functions - shows that speaker-aware simulation better aligns with real conversational patterns than the baseline method, capturing fine-grained temporal dependencies and realistic speaker alternation, while revealing open challenges in modeling long-range conversational structure.