RealMaster: Lifting Rendered Scenes into Photorealistic Video
Not on the map. Why?
Abstract (arXiv)
State-of-the-art video generation models produce remarkable photorealism, but they lack the precise control required to align generated content with specific scene requirements. Furthermore, without an underlying explicit geometry, these models cannot guarantee 3D consistency. Conversely, 3D engines offer granular control over every scene element and provide native 3D consistency by design, yet their output often remains trapped in the "uncanny valley". Bridging this sim-to-real gap requires both structural precision, where the output must exactly preserve the geometry and dynamics of the input, and global semantic transformation, where materials, lighting, and textures must be holistically transformed to achieve photorealism. We present RealMaster, a method that leverages video diffusion models to lift rendered video into photorealistic video while maintaining full alignment with the output of the 3D engine. To train this model, we generate a paired dataset via an anchor-based propagation strategy, where the first and last frames are enhanced for realism and propagated across the intermediate frames using geometric conditioning cues. We then train an IC-LoRA on these paired videos to distill the high-quality outputs of the pipeline into a model that generalizes beyond the pipeline's constraints, handling objects and characters that appear mid-sequence and enabling inference without requiring anchor frames. Evaluated on complex GTA-V sequences, RealMaster significantly outperforms existing video editing baselines, improving photorealism while preserving the geometry, dynamics, and identity specified by the original 3D control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
28 of 48 references.
References · 28
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2025Qwen-Image Technical Reportcited by 973
- 2017Unsupervised Image-to-Image Translation Networkscited by 602
- 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2018Learning to See in the Darkcited by 459
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
16 more
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
- 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
- 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
- 2023Pix2Video: Video Editing using Image Diffusioncited by 318
- 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
- 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
- 2023Dreamix: Video Diffusion Models are General Video Editorscited by 189
- 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editingcited by 165
- 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
- 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Modelscited by 135
- 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
- 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 76
- 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
- 2024Video Editing via Factorized Diffusion Distillationcited by 40
- 2025Zero-shot Synthetic Video Realism Enhancement via Structure-aware Denoisingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Realiz3D: 3D Generation Made Photorealistic via Domain-Aware Learning
- 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
- 2026TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos
- 2025View-Consistent Diffusion Representations for 3D-Consistent Video Generation
- 2026FixAnything: 3D-Consistent Rendering Refinement via Video Generative Priors
- 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
- 2026Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering