# WeatherEdit: Controllable Weather Editing with 4D Gaussian Field

arXiv: https://arxiv.org/abs/2505.20471 · HTML: https://mlsanity.com/paper/2505.20471/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation](https://mlsanity.com/paper/2212.01322/) (2022)
- [OmniRe: Omni Urban Scene Reconstruction](https://mlsanity.com/paper/2408.16760/) (2024)
- [A Closed-form Solution to Photorealistic Image Stylization](https://mlsanity.com/paper/1802.06474/) (2018)
- [One-Step Image Translation with Text-to-Image Models](https://mlsanity.com/paper/2403.12036/) (2024)
- [NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects](https://mlsanity.com/paper/2303.14435/) (2023)
- [HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting](https://mlsanity.com/paper/2311.17061/) (2023)
- [TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models](https://mlsanity.com/paper/2408.00735/) (2024)
- [ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field](https://mlsanity.com/paper/2211.13226/) (2022)
- [MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty](https://mlsanity.com/paper/2401.12761/) (2024)
- [Semantic Understanding of Foggy Scenes with Purely Synthetic Data](https://mlsanity.com/paper/1910.03997/) (2019)
- [ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods](https://mlsanity.com/paper/2110.02871/) (2021)
- [WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting](https://mlsanity.com/paper/2412.18862/) (2024)
- [Visualizing the Consequences of Climate Change Using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1905.03709/) (2019)
- [Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal](https://mlsanity.com/paper/2403.07684/) (2024)

## Built on by (most cited first, 12 of 12)

- [RealX3D: A Physically-Degraded 3D Benchmark for Multi-view Visual Restoration and Reconstruction](https://mlsanity.com/paper/2512.23437/) (2025)
- [AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing](https://mlsanity.com/paper/2603.26546/) (2026)
- [LoViF 2026 The First Challenge on Weather Removal in Videos](https://mlsanity.com/paper/2604.10655/) (2026)
- [Holo-World: Unified Camera, Object and Weather Control for Video World Model](https://mlsanity.com/paper/2606.20083/) (2026)
- [IntrinsicWeather: Controllable Weather Editing in Intrinsic Space](https://mlsanity.com/paper/2508.06982/) (2025)
- [WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation](https://mlsanity.com/paper/2602.22096/) (2026)
- [Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis](https://mlsanity.com/paper/2606.29020/) (2026)
- [muSync-GS: Physics-Synchronized Driving Video Synthesis for Weather and Geometric Road Hazards](https://mlsanity.com/paper/2608.04412/) (2026)
- [Physics-aware Masked Diffusion-based Flood Simulation for Urban Fisheye Disaster Detection](https://mlsanity.com/paper/2607.15527/) (2026)
- [Multi-Sensor Alignment for Weather Simulations](https://mlsanity.com/paper/2607.25612/) (2026)
- [GSRAIN: Physically Calibrated High-/Low-Frequency Rainfall Synthesis for 3D Gaussian Driving Scenes](https://mlsanity.com/paper/2608.02177/) (2026)
- [MeteoVerse: Unified Weather-Controllable Video World Model](https://mlsanity.com/paper/2609.36810/) (2026)

## Abstract (arXiv)

In this work, we present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects with controllable types and severity in 3D scenes. Our approach is structured into two key components: weather background editing and weather particle construction. For weather background editing, we introduce an all-in-one adapter that integrates multiple weather styles into a single pretrained diffusion model, enabling the generation of diverse weather effects in 2D image backgrounds. During inference, we design a Temporal-View (TV-) attention mechanism that follows a specific order to aggregate temporal and spatial information, ensuring consistent editing across multi-frame and multi-view images. To construct the weather particles, we first reconstruct a 3D scene using the edited images and then introduce a dynamic 4D Gaussian field to generate snowflakes, raindrops and fog in the scene. The attributes and dynamics of these particles are precisely controlled through physical-based modelling and simulation, ensuring realistic weather representation and flexible severity adjustments. Finally, we integrate the 4D Gaussian field with the 3D scene to render consistent and highly realistic weather effects. Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity, highlighting its potential for autonomous driving simulation in adverse weather. See project page: https://jumponthemoon.github.io/w-edit