mlsanityData to 7 Oct 2026

Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation

arXiv 2606.02441 ↗First posted 1 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Identity-preserving video generation (IPVG) aims to synthesize high-fidelity videos that follow text prompts while faithfully preserving a reference identity. Despite recent progress, existing IPVG methods still struggle to balance high-level semantic control and low-level identity fidelity. To bridge this gap, we propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-video generation. At the framework level, ST-DRC performs latent in-context feature injection by encoding the reference image with the video VAE and concatenating it with noisy video latents, enabling rich low-level identity details to be accessed without additional adapters. To separate identity-aware reference retrieval from appearance copying, we introduce TASS-RoPE, a Temporal-Adjacent Spatial-Shifted RoPE scheme that places reference tokens near the video sequence in time but shifts them in space, allowing reference information to flow through spatio-temporal attention while suppressing pixel-level copy-paste shortcuts. To further prevent shortcut learning and strengthen the otherwise diluted identity supervision in the diffusion objective, we combine appearance-invariant reference augmentation with face-guided identity objectives, encouraging the model to preserve identity under variations in color, pose, and layout. At inference time, we introduce a three-stream reference classifier-free guidance strategy that independently controls text adherence and reference fidelity. Experiments demonstrate that ST-DRC achieves strong identity preservation, prompt alignment, temporal consistency, and video quality with a lightweight design built on LTX-2.3. Our method ranks among the top submissions in the facial identity-preserving video generation track, validating the effectiveness of spatial-temporal decoupled reference conditioning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

42 of 61 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  9. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  10. 2025VACE: All-in-One Video Creation and Editingcited by 467
  11. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  12. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
30 more
  1. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  2. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  3. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
  4. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
  5. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
  6. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 84
  7. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
  8. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
  9. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  10. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  11. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordancecited by 60
  12. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 46
  13. 2025HuMo: Human-Centric Video Generation via Collaborative Multi-Modal Conditioningcited by 44
  14. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 39
  15. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 32
  16. 2025Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Promptscited by 28
  17. 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 22
  18. 2025Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergycited by 19
  19. 2025UltraGen: High-Resolution Video Generation with Hierarchical Attentioncited by 17
  20. 2023VideoDreamer: Customized Multi-Subject Text-to-Video Generation with Disen-Mix Finetuning on Language-Video Foundation Modelscited by 16
  21. 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
  22. 2025Kaleido: Open-Sourced Multi-Subject Reference Video Generation Modelcited by 16
  23. 2025EchoVideo: Identity-Preserving Human Video Generation by Multimodal Feature Fusioncited by 12
  24. 2025First Frame Is the Place to Go for Video Content Customizationcited by 9
  25. 2025SMRABooth: Subject and Motion Representation Alignment for Customized Video Generationcited by 9
  26. 2025Identity-Preserving Text-to-Video Generation via Training-Free Prompt, Image, and Guidance Enhancementcited by 8
  27. 2025ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generationcited by 5
  28. 2025Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representationscited by 4
  29. 2025OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulationcited by 4
  30. 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2

Built on by

Not cited yet.