# NeRF-HuGS: Improved Neural Radiance Fields in Non-static Scenes Using Heuristics-Guided Segmentation

arXiv: https://arxiv.org/abs/2403.17537 · HTML: https://mlsanity.com/paper/2403.17537/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 58 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 30 of 47)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs](https://mlsanity.com/paper/2112.10703/) (2021)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)

## Built on by (most cited first, 25 of 25)

- [SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting](https://mlsanity.com/paper/2406.20055/) (2024)
- [DreamDrive: Generative 4D Scene Modeling from Street View Images](https://mlsanity.com/paper/2501.00601/) (2024)
- [DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery](https://mlsanity.com/paper/2503.16964/) (2025)
- [HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting](https://mlsanity.com/paper/2412.03844/) (2024)
- [DeGauss: Dynamic-Static Decomposition with Gaussian Splatting for Distractor-free 3D Reconstruction](https://mlsanity.com/paper/2503.13176/) (2025)
- [DeSplat: Decomposed Gaussian Splatting for Distractor-Free Rendering](https://mlsanity.com/paper/2411.19756/) (2024)
- [T-3DGS: Removing Transient Objects for 3D Scene Reconstruction](https://mlsanity.com/paper/2412.00155/) (2024)
- [RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS](https://mlsanity.com/paper/2506.02751/) (2025)
- [SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction](https://mlsanity.com/paper/2411.12592/) (2024)
- [Distractor-free Generalizable 3D Gaussian Splatting](https://mlsanity.com/paper/2411.17605/) (2024)
- [EgoSplat: Open-Vocabulary Egocentric Scene Understanding with Language Embedded 3D Gaussian Splatting](https://mlsanity.com/paper/2503.11345/) (2025)
- [WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments](https://mlsanity.com/paper/2601.10716/) (2026)
- [MVGSR: Multi-View Consistency Gaussian Splatting for Robust Surface Reconstruction](https://mlsanity.com/paper/2503.08093/) (2025)
- [A Constrained Optimization Approach for Gaussian Splatting from Coarsely-posed Images and Noisy Lidar Point Clouds](https://mlsanity.com/paper/2504.09129/) (2025)
- [OCSplats: Observation Completeness Quantification and Label Noise Separation in 3DGS](https://mlsanity.com/paper/2508.01239/) (2025)
- [WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild](https://mlsanity.com/paper/2608.28240/) (2026)
- [WildGHand: Learning Anti-Perturbation Gaussian Hand Avatars from Monocular In-the-Wild Videos](https://mlsanity.com/paper/2602.20556/) (2026)
- [Sparse View Distractor-Free Gaussian Splatting](https://mlsanity.com/paper/2603.01603/) (2026)
- [3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification](https://mlsanity.com/paper/2603.07587/) (2026)
- [FullCircle: Effortless 3D Reconstruction from Casual 360^circ Captures](https://mlsanity.com/paper/2603.22572/) (2026)
- [DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis](https://mlsanity.com/paper/2604.13416/) (2026)
- [MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene](https://mlsanity.com/paper/2604.17965/) (2026)
- [Difix3D-W: Distractor-Free Few-Shot 3D Gaussian Splatting in the Wild](https://mlsanity.com/paper/2604.27422/) (2026)
- [HarmoGS: Robust 3D Gaussian Splatting in the Wild via Conflict-Aware Gradient Harmonization](https://mlsanity.com/paper/2605.13073/) (2026)
- [Rectifying Mask via Entropy for Distractor-Free 3DGS in Ambiguous Scenarios](https://mlsanity.com/paper/2606.29496/) (2026)

## Abstract (arXiv)

Neural Radiance Field (NeRF) has been widely recognized for its excellence in novel view synthesis and 3D scene reconstruction. However, their effectiveness is inherently tied to the assumption of static scenes, rendering them susceptible to undesirable artifacts when confronted with transient distractors such as moving objects or shadows. In this work, we propose a novel paradigm, namely "Heuristics-Guided Segmentation" (HuGS), which significantly enhances the separation of static scenes from transient distractors by harmoniously combining the strengths of hand-crafted heuristics and state-of-the-art segmentation models, thus significantly transcending the limitations of previous solutions. Furthermore, we delve into the meticulous design of heuristics, introducing a seamless fusion of Structure-from-Motion (SfM)-based heuristics and color residual heuristics, catering to a diverse range of texture profiles. Extensive experiments demonstrate the superiority and robustness of our method in mitigating transient distractors for NeRFs trained in non-static scenes. Project page: https://cnhaox.github.io/NeRF-HuGS/.