mlsanityData to 14 Oct 2026

SoulX-LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memory

arXiv 2603.11746 ↗First posted 12 Mar 2026cs.CVcited by 2

On the map: Diffusion models → Video generation

Abstract (arXiv)

Autoregressive (AR) diffusion models offer a promising framework for sequential generation tasks such as video synthesis by combining diffusion modeling with causal inference. Although they support streaming generation, existing AR diffusion methods struggle to scale efficiently. In this paper, we identify two key challenges in hour-scale real-time human animation. First, most forcing strategies propagate sample-level representations with mismatched diffusion states, causing inconsistent learning signals and unstable convergence. Second, historical representations grow unbounded and lack structure, preventing effective reuse of cached states and severely limiting inference efficiency. To address these challenges, we propose Neighbor Forcing, a diffusion-step-consistent AR formulation that propagates temporally adjacent frames as latent neighbors under the same noise condition. This design provides a distribution-aligned and stable learning signal while preserving drifting throughout the AR chain. Building upon this, we introduce a structured ConvKV memory mechanism that compresses the keys and values in causal attention into a fixed-length representation, enabling constant-memory inference and truly infinite video generation without relying on short-term motion-frame memory. Extensive experiments demonstrate that our approach significantly improves training convergence, hour-scale generation quality, and inference efficiency compared to existing AR diffusion methods. Numerically, LiveAct enables hour-scale real-time human animation and supports 20 FPS real-time streaming inference on as few as two NVIDIA H100 or H200 GPUs. Quantitative results demonstrate that our method attains state-of-the-art performance in lip-sync accuracy, human animation quality, and emotional expressiveness, with the lowest inference cost.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 31 of its 37 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

31 of 37 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resampling17 Dec
  2. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generation2 Oct
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  4. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective11 Jul
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion9 Jun
  6. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models17 Apr
  7. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion10 Mar
  8. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
  9. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing25 Nov
  10. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scale26 Oct
  11. 2024Progressive Autoregressive Video Diffusion Models10 Oct
  12. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar

Other references · 19

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  6. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  8. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  9. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  10. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  11. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
  12. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
7 more
  1. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  2. 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 83
  3. 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 72
  4. 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 41
  5. 2025Human Motion Video Generation: A Surveycited by 25
  6. 2025Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generationcited by 16
  7. 2025Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregressioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  2. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
  3. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory
  4. 2026Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model
  5. 2025Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation
  6. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation
  7. 2025Generative Pre-trained Autoregressive Diffusion Transformer
  8. 2026LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time