mlsanityData to 14 Oct 2026

Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh

arXiv 2608.00094 ↗First posted 30 Jul 2026cs.CVcited by 2

On the map: Diffusion models → Video generation

Abstract (arXiv)

Pretrained video diffusion models can act as renderers when the desired scene state is already specified by an animated mesh, a camera trajectory, and a reference image. This 4D generative rendering setting raises a representation question: what image-format condition lets a video backbone obey both camera motion and scene-internal animation? We propose DAR, a reference-guided renderer that extends Wan2.2 camera control from Pl"ucker rays alone to a joint camera-plus-geometry interface. DAR projects a neural 4D G-buffer (tracking, world position, and normal) from the animated mesh and injects it through a widened control adapter while preserving the pretrained image-to-video prior. The central design choice is the pair of tracking and world position. Tracking identifies the persistent surface element that should carry appearance; world position gives its current scene-coordinate state; normal supplies local shape. Depth plus calibrated rays can recover 3D in principle, but depth is a camera-dependent chart in which camera and object motion are mixed. On the 68-case DAR-4D benchmark, LoRA DAR reaches PSNR 23.22, SSIM 0.895, and LPIPS 0.134, improving over off-the-shelf Wan2.2-Depth by 1.54 dB PSNR; a full fine-tune reaches PSNR 25.36 and SSIM 0.917. Matched ablations show that replacing world position by depth reduces PSNR by 1.26--1.55 dB at every checkpoint, supporting tracking+world-position correspondence as a practical 4D rendering condition.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 31 of its 49 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

31 of 49 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Models10 Jun
  2. 2025VACE: All-in-One Video Creation and Editing10 Mar
  3. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control7 Jan
  4. 2023Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models3 Dec

Other references · 27

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  3. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  4. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  5. 2022Kubric: A scalable dataset generatorcited by 381
  6. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  7. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  8. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 148
  9. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 124
  10. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 123
  11. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  12. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 46
15 more
  1. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 40
  2. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
  3. 2024Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusioncited by 15
  4. 2024Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputscited by 12
  5. 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturingcited by 10
  6. 2026Feedforward 3D Editing Learns from Semantic-Part Transformationcited by 10
  7. 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 9
  8. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  9. 2026HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesiscited by 7
  10. 2026Animator-Centric Skeleton Generation on Objects with Fine-Grained Detailscited by 7
  11. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
  12. 2026Engine-Native Editable 3D World Reconstruction with Objects and Lightingcited by 6
  13. 2025DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modelingcited by 5
  14. 2026LottieGPT: Tokenizing Vector Animation for Autoregressive Generationcited by 5
  15. 2026PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 20264Director: Controlling Video World Models with Rigid 3D Geometry
  2. 20264DStreamCtrl: Interactive Video Generation with Online 4D Control
  3. 2025Tracking-Guided 4D Generation: Foundation-Tracker Motion Priors for 3D Model Animation
  4. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Control
  5. 2024Controlling Space and Time with Diffusion Models
  6. 2026Beyond Inpainting: Unleash 3D Understanding for Precise Camera-Controlled Video Generation
  7. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
  8. 2026Demiurge: Forging Navigable 4D Worlds via Autoregressive Video Rerendering