DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing
Not on the map. Why?
Abstract (arXiv)
We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly, including tempo shifts, drops, or user edits. Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio-motion alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 60 of its 71 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
60 of 71 references.
References · 60
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2017Neural Discrete Representation Learningcited by 4,712
- 2022Video Diffusion Modelscited by 2,305
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
- 2022Human Motion Diffusion Modelcited by 1,080
38 more
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
- 2022Autoregressive Image Generation using Residual Quantizationcited by 722
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
- 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
- 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
- 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
- 2023Photorealistic Video Generation with Diffusion Modelscited by 306
- 2022EDGE: Editable Dance Generation From Musiccited by 287
- 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
- 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
- 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
- 2025History-Guided Video Diffusioncited by 171
- 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
- 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
- 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 137
- 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
- 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
- 2024Rolling Diffusion Modelscited by 104
- 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 88
- 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
- 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
- 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
- 2025Retargeting Matters: General Motion Retargeting for Humanoid Motion Trackingcited by 75
- 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 61
- 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
- 2023TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesiscited by 49
- 2023PADL: Language-Directed Physics-Based Character Controlcited by 48
- 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
- 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 28
- 2025UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Controlcited by 25
- 2024Beat-It: Beat-Synchronized Multi-Condition 3D Dance Generationcited by 24
- 2024Scaling Large Motion Models with Million-Level Human Motionscited by 22
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.