mlsanityData to 14 Oct 2026

Generative World Renderer

arXiv 2604.02329 ↗First posted 2 Apr 2026cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 128 of its 163 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

128 of 163 references.

References · 128

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  12. 2024Depth Anything V2cited by 1,422
38 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  2. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  3. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  4. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  6. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  7. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  8. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  9. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  10. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  11. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  12. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  13. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  14. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  15. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  16. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  17. 2021Advances in Neural Renderingcited by 365
  18. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  19. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  20. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 327
  21. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 326
  22. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
  23. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  24. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 294
  25. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  26. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  27. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  28. 2018Learning Blind Video Temporal Consistencycited by 253
  29. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  30. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 249
  31. 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 224
  32. 2020Consistent Video Depth Estimationcited by 220
  33. 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 213
  34. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 207
  35. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  36. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 203
  37. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  38. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026RGBX-Next: Towards Realistic Generative Rendering from G-Buffers
  2. 2026DreamX-World 1.0: A General-Purpose Interactive World Model
  3. 2026Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering
  4. 2026SpatialCrafter: Single Image World Modeling with Generative 3D Proxies
  5. 2026A Hybrid Approach for Closing the Sim2real Appearance Gap in Game Engine Synthetic Datasets
  6. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report
  7. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory
  8. 2026Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation