# READ: Large-Scale Neural Scene Rendering for Autonomous Driving

arXiv: https://arxiv.org/abs/2205.05509 · HTML: https://mlsanity.com/paper/2205.05509/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 23 of 23)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [Neural Rerendering in the Wild](https://mlsanity.com/paper/1904.04290/) (2019)
- [Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation](https://mlsanity.com/paper/1912.12215/) (2019)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Imitating Arbitrary Talking Style for Realistic Audio-DrivenTalking Face Synthesis](https://mlsanity.com/paper/2111.00203/) (2021)
- [Dual Attention GANs for Semantic Image Synthesis](https://mlsanity.com/paper/2008.13024/) (2020)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2107.01779/) (2021)
- [Brno Urban Dataset -- The New Data for Self-Driving Agents and Mapping Tasks](https://mlsanity.com/paper/1909.06897/) (2019)
- [iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering](https://mlsanity.com/paper/2108.05577/) (2021)
- [TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer](https://mlsanity.com/paper/2009.02819/) (2020)
- [Few-shot Neural Human Performance Rendering from Sparse RGBD Videos](https://mlsanity.com/paper/2107.06505/) (2021)

## Built on by (most cited first, 30 of 51)

- [Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting](https://mlsanity.com/paper/2401.01339/) (2024)
- [Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering](https://mlsanity.com/paper/2311.18561/) (2023)
- [StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views](https://mlsanity.com/paper/2306.04988/) (2023)
- [Human Gaussian Splatting: Real-time Rendering of Animatable Avatars](https://mlsanity.com/paper/2311.17113/) (2023)
- [Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents](https://mlsanity.com/paper/2402.05746/) (2024)
- [UniPAD: A Universal Pre-training Paradigm for Autonomous Driving](https://mlsanity.com/paper/2310.08370/) (2023)
- [Urban Radiance Field Representation with Deformable Neural Mesh Primitives](https://mlsanity.com/paper/2307.10776/) (2023)
- [Real-Time Neural Rasterization for Large Scenes](https://mlsanity.com/paper/2311.05607/) (2023)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives](https://mlsanity.com/paper/2404.09748/) (2024)
- [Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities](https://mlsanity.com/paper/2401.08045/) (2024)
- [UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets](https://mlsanity.com/paper/2403.05086/) (2024)
- [RoMe: Towards Large Scale Road Surface Reconstruction via Mesh Representation](https://mlsanity.com/paper/2306.11368/) (2023)
- [Neural Radiance Field in Autonomous Driving: A Survey](https://mlsanity.com/paper/2404.13816/) (2024)
- [AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis](https://mlsanity.com/paper/2402.17483/) (2024)
- [GGS: Generalizable Gaussian Splatting for Lane Switching in Autonomous Driving](https://mlsanity.com/paper/2409.02382/) (2024)
- [HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes](https://mlsanity.com/paper/2403.20032/) (2024)
- [MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulation](https://mlsanity.com/paper/2307.14981/) (2023)
- [From Chaos to Clarity: 3DGS in the Dark](https://mlsanity.com/paper/2406.08300/) (2024)
- [Applications of Large Scale Foundation Models for Autonomous Driving](https://mlsanity.com/paper/2311.12144/) (2023)
- [EMIE-MAP: Large-Scale Road Surface Reconstruction Based on Explicit Mesh and Implicit Encoding](https://mlsanity.com/paper/2403.11789/) (2024)
- [3D Data Augmentation for Driving Scenes on Camera](https://mlsanity.com/paper/2303.10340/) (2023)
- [Boosting Point Clouds Rendering via Radiance Mapping](https://mlsanity.com/paper/2210.15107/) (2022)
- [Frequency-Modulated Point Cloud Rendering with Easy Editing](https://mlsanity.com/paper/2303.07596/) (2023)
- [DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes](https://mlsanity.com/paper/2311.16664/) (2023)
- [Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2503.14537/) (2025)
- [Interactive Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2303.11537/) (2023)
- [HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios](https://mlsanity.com/paper/2310.05483/) (2023)
- [Efficient Depth-Guided Urban View Synthesis](https://mlsanity.com/paper/2407.12395/) (2024)
- [XScale-NVS: Cross-Scale Novel View Synthesis with Hash Featurized Manifold](https://mlsanity.com/paper/2403.19517/) (2024)

## Abstract (arXiv)

Synthesizing free-view photo-realistic images is an important task in multimedia. With the development of advanced driver assistance systems~(ADAS) and their applications in autonomous vehicles, experimenting with different scenarios becomes a challenge. Although the photo-realistic street scenes can be synthesized by image-to-image translation methods, which cannot produce coherent scenes due to the lack of 3D information. In this paper, a large-scale neural rendering method is proposed to synthesize the autonomous driving scene~(READ), which makes it possible to synthesize large-scale driving scenarios on a PC through a variety of sampling schemes. In order to represent driving scenarios, we propose an ω rendering network to learn neural descriptors from sparse point clouds. Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes. Experiments show that our model performs well in large-scale driving scenarios.