# Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech

arXiv: https://arxiv.org/abs/2512.21706 · HTML: https://mlsanity.com/paper/2512.21706/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 25 Dec 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs](https://mlsanity.com/paper/2404.07103/) (2024)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [Demystifying Chains, Trees, and Graphs of Thoughts](https://mlsanity.com/paper/2401.14295/) (2024)
- [Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models](https://mlsanity.com/paper/2507.23159/) (2025)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs](https://mlsanity.com/paper/2104.08400/) (2021)
- [Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoning](https://mlsanity.com/paper/2412.12175/) (2024)
- [Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection](https://mlsanity.com/paper/2312.12810/) (2023)
- [Towards Hierarchical Spoken Language Dysfluency Modeling](https://mlsanity.com/paper/2401.10015/) (2024)
- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [When can I Speak? Predicting initiation points for spoken dialogue agents](https://mlsanity.com/paper/2208.03812/) (2022)
- [Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection](https://mlsanity.com/paper/2410.15929/) (2024)
- [Class Distance Weighted Cross Entropy Loss for Classification of Disease Severity](https://mlsanity.com/paper/2412.01246/) (2024)
- [HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation](https://mlsanity.com/paper/2402.09390/) (2024)
- [EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems](https://mlsanity.com/paper/2508.17623/) (2025)
- [AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues](https://mlsanity.com/paper/2510.07355/) (2025)

## Built on by (most cited first, 1 of 1)

- [Agentic-DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech](https://mlsanity.com/paper/2608.16053/) (2026)

## Abstract (arXiv)

Human conversation is organized by an implicit chain of thoughts that manifests as timed speech acts. Capturing this causal pathway is key to building natural full-duplex interactive systems. We introduce a framework that enables reasoning over conversational behaviors by modeling this process as causal inference within a Graph-of-Thoughts (GoT). Our approach formalizes the intent-to-action pathway with a hierarchical labeling scheme, predicting high-level communicative intents and low-level speech acts to learn their causal and temporal dependencies. To train this system, we develop a hybrid corpus that pairs controllable, event-rich simulations with human-annotated rationales and real conversational speech. The GoT framework structures streaming predictions as an evolving graph, enabling a multimodal transformer to forecast the next speech act, generate concise justifications for its decisions, and dynamically refine its reasoning. Experiments on both synthetic and real duplex dialogues show that the framework delivers robust behavior detection, produces interpretable reasoning chains, and establishes a foundation for benchmarking conversational reasoning in full duplex spoken dialogue systems.