mlsanityData to 14 Oct 2026

DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing

arXiv 2605.28491 ↗First posted 27 May 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly, including tempo shifts, drops, or user edits. Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio-motion alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 60 of its 71 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

60 of 71 references.

References · 60

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2022Video Diffusion Modelscited by 2,305
  10. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  11. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  12. 2022Human Motion Diffusion Modelcited by 1,080
38 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  2. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  3. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  6. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  7. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  8. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  9. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  10. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  12. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  13. 2022EDGE: Editable Dance Generation From Musiccited by 287
  14. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
  15. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  16. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  17. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
  18. 2025History-Guided Video Diffusioncited by 171
  19. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  20. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  21. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 137
  22. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
  23. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
  24. 2024Rolling Diffusion Modelscited by 104
  25. 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 88
  26. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
  27. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  28. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  29. 2025Retargeting Matters: General Motion Retargeting for Humanoid Motion Trackingcited by 75
  30. 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 61
  31. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
  32. 2023TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesiscited by 49
  33. 2023PADL: Language-Directed Physics-Based Character Controlcited by 48
  34. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  35. 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 28
  36. 2025UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Controlcited by 25
  37. 2024Beat-It: Beat-Synchronized Multi-Condition 3D Dance Generationcited by 24
  38. 2024Scaling Large Motion Models with Million-Level Human Motionscited by 22

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.