mlsanityData to 14 Oct 2026

TC4D: Trajectory-Conditioned Text-to-4D Generation

arXiv 2403.17920 ↗First posted 26 Mar 2024cs.CVcited by 77

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent techniques for text-to-4D generation synthesize dynamic 3D scenes using supervision from pre-trained text-to-video models. However, existing representations for motion, such as deformation models or time-dependent neural representations, are limited in the amount of motion they can generate-they cannot synthesize motion extending far beyond the bounding box used for volume rendering. The lack of a more flexible motion model contributes to the gap in realism between 4D generation methods and recent, near-photorealistic video generation models. Here, we propose TC4D: trajectory-conditioned text-to-4D generation, which factors motion into global and local components. We represent the global motion of a scene's bounding box using rigid transformation along a trajectory parameterized by a spline. We learn local deformations that conform to the global trajectory using supervision from a text-to-video model. Our approach enables the synthesis of scenes animated along arbitrary trajectories, compositional scene generation, and significant improvements to the realism and amount of generated motion, which we evaluate qualitatively and through a user study. Video results can be viewed on our website: https://sherwinbahmani.github.io/tc4d.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). Builds on 112 of its 121 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

112 of 121 references.

References · 112

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
38 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  5. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  6. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  7. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  8. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  9. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  11. 2023ModelScope Text-to-Video Technical Reportcited by 658
  12. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  13. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  14. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  15. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  16. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  17. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  18. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 504
  19. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  20. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  21. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  22. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  23. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  24. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 370
  25. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
  26. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  27. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  28. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  29. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  30. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  31. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  32. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 287
  33. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  34. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  35. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  36. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  37. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  38. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211

50 most cited of 112 All 112 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 33; 2026: 21.

77 papers.

Sort:

By area:

  1. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  2. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  4. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  5. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
  6. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  7. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  8. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 56
  9. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  10. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  11. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  12. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
38 more
  1. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
  2. 2025Motion Anything: Any to Motion Generationcited by 42
  3. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  4. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  5. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 38
  6. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  7. 2025Dynamic Camera Poses and Where to Find Themcited by 27
  8. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  9. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  10. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  11. 2024Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesiscited by 19
  12. 2024MagicPose4D: Crafting Articulated Models with Appearance and Motion Controlcited by 18
  13. 2024Compositional 3D-aware Video Generation with LLM Directorcited by 18
  14. 2025Generative AI for Film Creation: A Survey of Recent Advancescited by 17
  15. 2025Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Modelscited by 17
  16. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 16
  17. 20244Dynamic: Text-to-4D Generation with Hybrid Priorscited by 15
  18. 2024ChatCam: Empowering Camera Control through Conversational AIcited by 15
  19. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 15
  20. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 14
  21. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  22. 2024Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generationcited by 12
  23. 20253D-Fixup: Advancing Photo Editing with 3D Priorscited by 12
  24. 2024ElastoGen: 4D Generative Elastodynamicscited by 10
  25. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  26. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 9
  27. 2026PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generationcited by 9
  28. 2025Towards Scalable Spatial Intelligence via 2D-to-3D Data Liftingcited by 8
  29. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creationcited by 8
  30. 2025AnimaMimic: Imitating 3D Animation from Video Priorscited by 7
  31. 2024Bringing Objects to Life: training-free 4D generation from 3D objects through view consistent noisecited by 5
  32. 2025GPS as a Control Signal for Image Generationcited by 5
  33. 2025SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Modelcited by 5
  34. 20254DSTR: Advancing Generative 4D Gaussians with Spatial-Temporal Rectification for High-Quality and Consistent 4D Generationcited by 5
  35. 2025Captain Safari: A World Engine with Pose-Aligned 3D Memorycited by 5
  36. 2025Animus3D: Text-driven 3D Animation via Motion Score Distillationcited by 5
  37. 2026Choreographing a World of Dynamic Objectscited by 5
  38. 2024GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objectscited by 4

50 most cited of 77 All 77 (JSON)