# Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh

arXiv: https://arxiv.org/abs/2608.00094 · HTML: https://mlsanity.com/paper/2608.00094/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 31)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [ReCamMaster: Camera-Controlled Generative Rendering from A Single Video](https://mlsanity.com/paper/2503.11647/) (2025)
- [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control](https://mlsanity.com/paper/2501.03847/) (2025)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)
- [4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface](https://mlsanity.com/paper/2105.01905/) (2021)
- [Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models](https://mlsanity.com/paper/2312.13913/) (2023)
- [Texture Generation on 3D Meshes with Point-UV Diffusion](https://mlsanity.com/paper/2308.10490/) (2023)
- [NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos](https://mlsanity.com/paper/2601.00393/) (2026)
- [DeepVerse: 4D Autoregressive Video Generation as a World Model](https://mlsanity.com/paper/2506.01103/) (2025)
- [Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models](https://mlsanity.com/paper/2312.01409/) (2023)
- [DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characters](https://mlsanity.com/paper/2411.17423/) (2024)
- [Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion](https://mlsanity.com/paper/2412.15050/) (2024)
- [Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs](https://mlsanity.com/paper/2404.04363/) (2024)
- [RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing](https://mlsanity.com/paper/2412.16778/) (2024)
- [Feedforward 3D Editing Learns from Semantic-Part Transformation](https://mlsanity.com/paper/2605.27351/) (2026)
- [From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction](https://mlsanity.com/paper/2602.01661/) (2026)
- [PhysInOne: Visual Physics Learning and Reasoning in One Suite](https://mlsanity.com/paper/2604.09415/) (2026)
- [HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesis](https://mlsanity.com/paper/2604.03305/) (2026)
- [Animator-Centric Skeleton Generation on Objects with Fine-Grained Details](https://mlsanity.com/paper/2604.20539/) (2026)
- [One Video, One World: Turning Monocular Video into Physical 4D Scenes](https://mlsanity.com/paper/2606.31388/) (2026)
- [Engine-Native Editable 3D World Reconstruction with Objects and Lighting](https://mlsanity.com/paper/2607.20889/) (2026)
- [DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling](https://mlsanity.com/paper/2512.03000/) (2025)
- [LottieGPT: Tokenizing Vector Animation for Autoregressive Generation](https://mlsanity.com/paper/2604.11792/) (2026)

## Built on by (most cited first, 2 of 2)

- [Code Plans, Diffusion Renders: Open-Ended Generative World Modeling](https://mlsanity.com/paper/2609.26458/) (2026)
- [Proxy2World: Learning to Generate Worlds From Lightweight Proxies without Seeing Them](https://mlsanity.com/paper/2609.35023/) (2026)

## Abstract (arXiv)

Pretrained video diffusion models can act as renderers when the desired scene state is already specified by an animated mesh, a camera trajectory, and a reference image. This 4D generative rendering setting raises a representation question: what image-format condition lets a video backbone obey both camera motion and scene-internal animation? We propose DAR, a reference-guided renderer that extends Wan2.2 camera control from Pl"ucker rays alone to a joint camera-plus-geometry interface. DAR projects a neural 4D G-buffer (tracking, world position, and normal) from the animated mesh and injects it through a widened control adapter while preserving the pretrained image-to-video prior. The central design choice is the pair of tracking and world position. Tracking identifies the persistent surface element that should carry appearance; world position gives its current scene-coordinate state; normal supplies local shape. Depth plus calibrated rays can recover 3D in principle, but depth is a camera-dependent chart in which camera and object motion are mixed. On the 68-case DAR-4D benchmark, LoRA DAR reaches PSNR 23.22, SSIM 0.895, and LPIPS 0.134, improving over off-the-shelf Wan2.2-Depth by 1.54 dB PSNR; a full fine-tune reaches PSNR 25.36 and SSIM 0.917. Matched ablations show that replacing world position by depth reduces PSNR by 1.26--1.55 dB at every checkpoint, supporting tracking+world-position correspondence as a practical 4D rendering condition.