# WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion

arXiv: https://arxiv.org/abs/2504.13561 · HTML: https://mlsanity.com/paper/2504.13561/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in World modeling from LiDAR generation, one of 11 directions in World models.
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Placed in

- World models / [World modeling from LiDAR generation](https://mlsanity.com/wm/world-modeling-from-lidar-generation/)
- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Diagonal State Spaces are as Effective as Structured State Spaces](https://mlsanity.com/paper/2203.14343/) (2022)
- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving](https://mlsanity.com/paper/1903.08701/) (2019)
- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding](https://mlsanity.com/paper/1808.01265/) (2018)
- [Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather](https://mlsanity.com/paper/2108.05249/) (2021)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation](https://mlsanity.com/paper/2311.01448/) (2023)
- [LiDAR Snowfall Simulation for Robust 3D Object Detection](https://mlsanity.com/paper/2203.15118/) (2022)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [Towards Realistic Scene Generation with LiDAR Diffusion Models](https://mlsanity.com/paper/2404.00815/) (2024)
- [Rain rendering for evaluating and improving robustness to bad weather](https://mlsanity.com/paper/2009.03683/) (2020)
- [Uncertainty Inspired RGB-D Saliency Detection](https://mlsanity.com/paper/2009.03075/) (2020)
- [LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.09256/) (2023)
- [RangeLDM: Fast Realistic LiDAR Point Cloud Generation](https://mlsanity.com/paper/2403.10094/) (2024)

## Built on by (most cited first, 12 of 12)

- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2508.03692/) (2025)
- [La La LiDAR: Large-Scale Layout Generation from LiDAR Data](https://mlsanity.com/paper/2508.03691/) (2025)
- [U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2512.02982/) (2025)
- [Veila: Panoramic LiDAR Generation from a Monocular RGB Image](https://mlsanity.com/paper/2508.03690/) (2025)
- [Learning to Generate 4D LiDAR Sequences](https://mlsanity.com/paper/2509.11959/) (2025)
- [Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments](https://mlsanity.com/paper/2604.07997/) (2026)
- [OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation](https://mlsanity.com/paper/2605.13815/) (2026)
- [GEM: Generating LiDAR World Model via Deformable Mamba](https://mlsanity.com/paper/2605.07326/) (2026)
- [MambaRain: Multi-Scale Mamba-Attention Framework for 0-3 Hour Precipitation Nowcasting](https://mlsanity.com/paper/2605.14606/) (2026)
- [LiAuto-GeoX: Efficient Grounded Driving Transformer](https://mlsanity.com/paper/2606.05774/) (2026)
- [T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation](https://mlsanity.com/paper/2606.30147/) (2026)

## Abstract (arXiv)

3D scene perception demands a large amount of adverse-weather LiDAR data, yet the cost of LiDAR data collection presents a significant scaling-up challenge. To this end, a series of LiDAR simulators have been proposed. Yet, they can only simulate a single adverse weather with a single physical model, and the fidelity of the generated data is quite limited. This paper presents WeatherGen, the first unified diverse-weather LiDAR data diffusion generation framework, significantly improving fidelity. Specifically, we first design a map-based data producer, which can provide a vast amount of high-quality diverse-weather data for training purposes. Then, we utilize the diffusion-denoising paradigm to construct a diffusion model. Among them, we propose a spider mamba generator to restore the disturbed diverse weather data gradually. The spider mamba models the feature interactions by scanning the LiDAR beam circle or central ray, excellently maintaining the physical structure of the LiDAR data. Subsequently, following the generator to transfer real-world knowledge, we design a latent feature aligner. Afterward, we devise a contrastive learning-based controller, which equips weather control signals with compact semantic knowledge through language supervision, guiding the diffusion model to generate more discriminative data. Extensive evaluations demonstrate the high generation quality of WeatherGen. Through WeatherGen, we construct the mini-weather dataset, promoting the performance of the downstream task under adverse weather conditions. Code is available: https://github.com/wuyang98/weathergen