# RealMaster: Lifting Rendered Scenes into Photorealistic Video

arXiv: https://arxiv.org/abs/2603.23462 · HTML: https://mlsanity.com/paper/2603.23462/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [Pix2Video: Video Editing using Image Diffusion](https://mlsanity.com/paper/2303.12688/) (2023)
- [Video-P2P: Video Editing with Cross-attention Control](https://mlsanity.com/paper/2303.04761/) (2023)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Dreamix: Video Diffusion Models are General Video Editors](https://mlsanity.com/paper/2302.01329/) (2023)
- [FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing](https://mlsanity.com/paper/2310.05922/) (2023)
- [AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks](https://mlsanity.com/paper/2403.14468/) (2024)
- [Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models](https://mlsanity.com/paper/2303.17599/) (2023)
- [CoDeF: Content Deformation Fields for Temporally Consistent Video Processing](https://mlsanity.com/paper/2308.07926/) (2023)
- [Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset](https://mlsanity.com/paper/2510.15742/) (2025)
- [InstructVid2Vid: Controllable Video Editing with Natural Language Instructions](https://mlsanity.com/paper/2305.12328/) (2023)
- [Video Editing via Factorized Diffusion Distillation](https://mlsanity.com/paper/2403.09334/) (2024)
- [Zero-shot Synthetic Video Realism Enhancement via Structure-aware Denoising](https://mlsanity.com/paper/2511.14719/) (2025)

## Built on by (most cited first, 1 of 1)

- [Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering](https://mlsanity.com/paper/2607.21848/) (2026)

## Abstract (arXiv)

State-of-the-art video generation models produce remarkable photorealism, but they lack the precise control required to align generated content with specific scene requirements. Furthermore, without an underlying explicit geometry, these models cannot guarantee 3D consistency. Conversely, 3D engines offer granular control over every scene element and provide native 3D consistency by design, yet their output often remains trapped in the "uncanny valley". Bridging this sim-to-real gap requires both structural precision, where the output must exactly preserve the geometry and dynamics of the input, and global semantic transformation, where materials, lighting, and textures must be holistically transformed to achieve photorealism. We present RealMaster, a method that leverages video diffusion models to lift rendered video into photorealistic video while maintaining full alignment with the output of the 3D engine. To train this model, we generate a paired dataset via an anchor-based propagation strategy, where the first and last frames are enhanced for realism and propagated across the intermediate frames using geometric conditioning cues. We then train an IC-LoRA on these paired videos to distill the high-quality outputs of the pipeline into a model that generalizes beyond the pipeline's constraints, handling objects and characters that appear mid-sequence and enabling inference without requiring anchor frames. Evaluated on complex GTA-V sequences, RealMaster significantly outperforms existing video editing baselines, improving photorealism while preserving the geometry, dynamics, and identity specified by the original 3D control.