# ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field

arXiv: https://arxiv.org/abs/2312.09095 · HTML: https://mlsanity.com/paper/2312.09095/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes](https://mlsanity.com/paper/2104.06935/) (2021)
- [InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering](https://mlsanity.com/paper/2112.15399/) (2021)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [Is Attention All That NeRF Needs?](https://mlsanity.com/paper/2207.13298/) (2022)
- [Layer-structured 3D Scene Inference via View Synthesis](https://mlsanity.com/paper/1807.10264/) (2018)
- [SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutions](https://mlsanity.com/paper/2309.03955/) (2023)
- [Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network](https://mlsanity.com/paper/2012.15020/) (2020)
- [Unpaired Image Enhancement with Quality-Attention Generative Adversarial Network](https://mlsanity.com/paper/2012.15052/) (2020)

## Built on by (most cited first, 11 of 11)

- [Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2408.13912/) (2024)
- [TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers](https://mlsanity.com/paper/2408.13770/) (2024)
- [Perceptual-GS: Scene-adaptive Perceptual Densification for Gaussian Splatting](https://mlsanity.com/paper/2506.12400/) (2025)
- [MarkSplatter: Generalizable Watermarking for 3D Gaussian Splatting Model via Splatter Image Structure](https://mlsanity.com/paper/2509.00757/) (2025)
- [Stereo-GS: Multi-View Stereo Vision Model for Generalizable 3D Gaussian Splatting Reconstruction](https://mlsanity.com/paper/2507.14921/) (2025)
- [Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting](https://mlsanity.com/paper/2605.00408/) (2026)
- [SGCNeRF: Few-Shot Neural Rendering via Sparse Geometric Consistency Guidance](https://mlsanity.com/paper/2404.00992/) (2024)
- [Simple-RF: Regularizing Sparse Input Radiance Fields with Simpler Solutions](https://mlsanity.com/paper/2404.19015/) (2024)
- [Cooperative Perception: A Resource-Efficient Framework for Multi-Drone 3D Scene Reconstruction Using Federated Diffusion and NeRF](https://mlsanity.com/paper/2508.00967/) (2025)
- [Explicit Context-Driven Neural Acoustic Modeling for High-Fidelity RIR Generation](https://mlsanity.com/paper/2509.15210/) (2025)
- [MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene](https://mlsanity.com/paper/2604.17965/) (2026)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) have demonstrated impressive potential in synthesizing novel views from dense input, however, their effectiveness is challenged when dealing with sparse input. Existing approaches that incorporate additional depth or semantic supervision can alleviate this issue to an extent. However, the process of supervision collection is not only costly but also potentially inaccurate, leading to poor performance and generalization ability in diverse scenarios. In our work, we introduce a novel model: the Collaborative Neural Radiance Fields (ColNeRF) designed to work with sparse input. The collaboration in ColNeRF includes both the cooperation between sparse input images and the cooperation between the output of the neural radiation field. Through this, we construct a novel collaborative module that aligns information from various views and meanwhile imposes self-supervised constraints to ensure multi-view consistency in both geometry and appearance. A Collaborative Cross-View Volume Integration module (CCVI) is proposed to capture complex occlusions and implicitly infer the spatial location of objects. Moreover, we introduce self-supervision of target rays projected in multiple directions to ensure geometric and color consistency in adjacent regions. Benefiting from the collaboration at the input and output ends, ColNeRF is capable of capturing richer and more generalized scene representation, thereby facilitating higher-quality results of the novel view synthesis. Extensive experiments demonstrate that ColNeRF outperforms state-of-the-art sparse input generalizable NeRF methods. Furthermore, our approach exhibits superiority in fine-tuning towards adapting to new scenes, achieving competitive performance compared to per-scene optimized NeRF-based methods while significantly reducing computational costs. Our code is available at: https://github.com/eezkni/ColNeRF.