# MixNeRF: Modeling a Ray with Mixture Density for Novel View Synthesis from Sparse Inputs

arXiv: https://arxiv.org/abs/2302.08788 · HTML: https://mlsanity.com/paper/2302.08788/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 48 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Dense Depth Priors for Neural Radiance Fields from Sparse Input Views](https://mlsanity.com/paper/2112.03288/) (2021)
- [GeoNeRF: Generalizing NeRF with Geometry Priors](https://mlsanity.com/paper/2111.13539/) (2021)
- [Neural Rays for Occlusion-aware Image-based Rendering](https://mlsanity.com/paper/2107.13421/) (2021)
- [Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes](https://mlsanity.com/paper/2104.06935/) (2021)
- [GRF: Learning a General Radiance Field for 3D Representation and Rendering](https://mlsanity.com/paper/2010.04595/) (2020)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering](https://mlsanity.com/paper/2112.15399/) (2021)
- [SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image](https://mlsanity.com/paper/2204.00928/) (2022)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [Neural Lumigraph Rendering](https://mlsanity.com/paper/2103.11571/) (2021)

## Built on by (most cited first, 30 of 40)

- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutions](https://mlsanity.com/paper/2309.03955/) (2023)
- [Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction](https://mlsanity.com/paper/2305.15171/) (2023)
- [Decompositional Neural Scene Reconstruction with Generative Diffusion Prior](https://mlsanity.com/paper/2503.14830/) (2025)
- [FlipNeRF: Flipped Reflection Rays for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2306.17723/) (2023)
- [4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization](https://mlsanity.com/paper/2411.08879/) (2024)
- [ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretraining](https://mlsanity.com/paper/2312.09249/) (2023)
- [SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization](https://mlsanity.com/paper/2407.14257/) (2024)
- [FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors](https://mlsanity.com/paper/2410.16271/) (2024)
- [Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2411.03637/) (2024)
- [Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis?](https://mlsanity.com/paper/2403.06092/) (2024)
- [GSemSplat: Generalizable Semantic 3D Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2412.16932/) (2024)
- [HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video](https://mlsanity.com/paper/2510.05560/) (2025)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [Sources of Uncertainty in 3D Scene Reconstruction](https://mlsanity.com/paper/2409.06407/) (2024)
- [Global Latent Neural Rendering](https://mlsanity.com/paper/2312.08338/) (2023)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Models](https://mlsanity.com/paper/2309.07986/) (2023)
- [Fast Sun-aligned Outdoor Scene Relighting based on TensoRF](https://mlsanity.com/paper/2311.03965/) (2023)
- [Efficient Depth-Guided Urban View Synthesis](https://mlsanity.com/paper/2407.12395/) (2024)
- [SSNeRF: Sparse View Semi-supervised Neural Radiance Fields with Augmentation](https://mlsanity.com/paper/2408.09144/) (2024)
- [MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields](https://mlsanity.com/paper/2410.11394/) (2024)
- [Few-shot NeRF by Adaptive Rendering Loss Regularization](https://mlsanity.com/paper/2410.17839/) (2024)
- [CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency](https://mlsanity.com/paper/2402.16407/) (2024)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [Evidential Neural Radiance Fields](https://mlsanity.com/paper/2602.23574/) (2026)
- [Instant Uncertainty Calibration of NeRFs Using a Meta-Calibrator](https://mlsanity.com/paper/2312.02350/) (2023)
- [HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs](https://mlsanity.com/paper/2401.11711/) (2024)
- [FrameNeRF: A Simple and Efficient Framework for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2402.14586/) (2024)

## Abstract (arXiv)

Neural Radiance Field (NeRF) has broken new ground in the novel view synthesis due to its simple concept and state-of-the-art quality. However, it suffers from severe performance degradation unless trained with a dense set of images with different camera poses, which hinders its practical applications. Although previous methods addressing this problem achieved promising results, they relied heavily on the additional training resources, which goes against the philosophy of sparse-input novel-view synthesis pursuing the training efficiency. In this work, we propose MixNeRF, an effective training strategy for novel view synthesis from sparse inputs by modeling a ray with a mixture density model. Our MixNeRF estimates the joint distribution of RGB colors along the ray samples by modeling it with mixture of distributions. We also propose a new task of ray depth estimation as a useful training objective, which is highly correlated with 3D scene geometry. Moreover, we remodel the colors with regenerated blending weights based on the estimated ray depth and further improves the robustness for colors and viewpoints. Our MixNeRF outperforms other state-of-the-art methods in various standard benchmarks with superior efficiency of training and inference.