# GDPO-Listener: Expressive Interactive Head Generation via Auto-Regressive Flow Matching and Group reward-Decoupled Policy Optimization

arXiv: https://arxiv.org/abs/2603.25020 · HTML: https://mlsanity.com/paper/2603.25020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 33 of its 39 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 33)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization](https://mlsanity.com/paper/2601.05242/) (2026)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion](https://mlsanity.com/paper/2204.08451/) (2022)
- [Image and Video Tokenization with Binary Spherical Quantization](https://mlsanity.com/paper/2406.07548/) (2024)
- [VideoDPO: Omni-Preference Alignment for Video Diffusion Generation](https://mlsanity.com/paper/2412.14167/) (2024)
- [From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations](https://mlsanity.com/paper/2401.01885/) (2024)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors](https://mlsanity.com/paper/2212.04248/) (2022)
- [Can Language Models Learn to Listen?](https://mlsanity.com/paper/2308.10897/) (2023)
- [INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations](https://mlsanity.com/paper/2412.04037/) (2024)
- [DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2506.03517/) (2025)
- [Dyadic Interaction Modeling for Social Behavior Generation](https://mlsanity.com/paper/2403.09069/) (2024)
- [DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations](https://mlsanity.com/paper/2505.18096/) (2025)
- [Taming Teacher Forcing for Masked Autoregressive Video Generation](https://mlsanity.com/paper/2501.12389/) (2025)
- [ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model](https://mlsanity.com/paper/2502.20323/) (2025)
- [CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation](https://mlsanity.com/paper/2403.00274/) (2024)
- [Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversation](https://mlsanity.com/paper/2601.00664/) (2026)
- [UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speaking](https://mlsanity.com/paper/2512.09327/) (2025)

## Abstract (arXiv)

Generating realistic 3D head motion for dyadic interactions is a significant challenge in virtual human synthesis. While recent methods achieve impressive results with speaking heads, they frequently suffer from the `Regression-to-the-Mean' problem in listener motions, collapsing into static faces, and lack the parameter space for complex nonverbal motions. In this paper, we propose GDPO-Listener, a novel framework that achieves highly expressive speaking and listening motion generation. First, we introduce an Auto-Regressive Flow Matching architecture enabling stable supervised learning. Second, to overcome kinematic stillness, we apply the Group reward-Decoupled Policy Optimization (GDPO). By isolating reward normalization across distinct FLAME parameter groups, GDPO explicitly incentivizes high variance expressive generations. Finally, we enable explicit semantic text control for customizable responses. Extensive evaluations across the Seamless Interaction and DualTalk datasets demonstrate superior performance compared to existing baselines on long-term kinematic variance, visual expressivity and semantic controllability.