mlsanityData to 14 Oct 2026

ShapeGen4D: Towards High Quality 4D Shape Generation from Videos

arXiv 2510.06208 ↗First posted 7 Oct 2025cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Video-conditioned 4D shape generation aims to recover time-varying 3D geometry and view-consistent appearance directly from an input video. In this work, we introduce a native video-to-4D shape generation framework that synthesizes a single dynamic 3D representation end-to-end from the video. Our framework introduces three key components based on large-scale pre-trained 3D models: (i) a temporal attention that conditions generation on all frames while producing a time-indexed dynamic representation; (ii) a time-aware point sampling and 4D latent anchoring that promote temporally consistent geometry and texture; and (iii) noise sharing across frames to enhance temporal stability. Our method accurately captures non-rigid motion, volume changes, and even topological transitions without per-frame optimization. Across diverse in-the-wild videos, our method improves robustness and perceptual fidelity and reduces failure modes compared with the baselines.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 58 of its 63 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

58 of 63 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  8. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
38 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  2. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  3. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  6. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
  7. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  8. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  9. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  10. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  11. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  12. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  13. 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
  14. 2023Text-To-4D Dynamic Scene Generationcited by 205
  15. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  16. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  17. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  18. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
  19. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
  20. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
  21. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 125
  22. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  23. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 115
  24. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 112
  25. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
  26. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  27. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  28. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 93
  29. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 89
  30. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
  31. 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
  32. 2025Improving the Diffusability of Autoencoderscited by 80
  33. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  34. 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
  35. 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformercited by 61
  36. 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
  37. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 56
  38. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 48

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 19.

21 papers.

Sort:

By area:

  1. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusioncited by 16
  2. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  3. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  4. 2026Feed-Forward 3D Scene Modeling: A Problem-Driven Perspectivecited by 6
  5. 2026Choreographing a World of Dynamic Objectscited by 5
  6. 2026Syn4D: A Multiview Synthetic 4D Datasetcited by 4
  7. 2026ViPS: Video-informed Pose Spaces for Auto-Rigged Meshescited by 3
  8. 2026Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformerscited by 3
  9. 2025SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generationcited by 2
  10. 2026Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wildcited by 2
  11. 2026MeshReGen: A Unified 3D Geometry Regeneration Frameworkcited by 1
  12. 2026Fast 4D Mesh Generation by Spatio-Temporal Attention Chainscited by 1
9 more
  1. 2026Helix4D: Complex 4D Mesh Generationcited by 1
  2. 2026UniMate: One Unified Model to Animate Diverse Skeletonscited by 1
  3. 2025PoseGAM: Robust Unseen Object Pose Estimation via Geometry-Aware Multi-View Reasoningcited by 0
  4. 2026UNICA: A Unified Neural Framework for Controllable 3D Avatarscited by 0
  5. 2026Velox: Learning Representations of 4D Geometry and Appearancecited by 0
  6. 2026SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapescited by 0
  7. 2026MORPHOS: Autoregressive 4D Generation with Temporal Structured Latentscited by 0
  8. 2026Controllable Dynamic 3D Shape Generation via 3D Trajectories and Textcited by 0
  9. 2026BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitivescited by 0