# Distributed NeRF Learning for Collaborative Multi-Robot Perception

arXiv: https://arxiv.org/abs/2409.20289 · HTML: https://mlsanity.com/paper/2409.20289/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 30 of 31)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM](https://mlsanity.com/paper/2304.14377/) (2023)
- [InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering](https://mlsanity.com/paper/2112.15399/) (2021)
- [Vision-Only Robot Navigation in a Neural Radiance World](https://mlsanity.com/paper/2110.00168/) (2021)
- [iSDF: Real-Time Neural Signed Distance Fields for Robot Perception](https://mlsanity.com/paper/2204.02296/) (2022)
- [Distributed Stochastic Gradient Tracking Methods](https://mlsanity.com/paper/1805.11454/) (2018)
- [VR-NeRF: High-Fidelity Virtualized Walkable Spaces](https://mlsanity.com/paper/2311.02542/) (2023)
- [GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction](https://mlsanity.com/paper/2206.14735/) (2022)
- [Renderable Neural Radiance Map for Visual Navigation](https://mlsanity.com/paper/2303.00304/) (2023)
- [CORE: Cooperative Reconstruction for Multi-Agent Perception](https://mlsanity.com/paper/2307.11514/) (2023)
- [CP-SLAM: Collaborative Neural Point-based SLAM System](https://mlsanity.com/paper/2311.08013/) (2023)
- [H2-Mapping: Real-time Dense Mapping Using Hierarchical Hybrid Representation](https://mlsanity.com/paper/2306.03207/) (2023)
- [CATNIPS: Collision Avoidance Through Neural Implicit Probabilistic Scenes](https://mlsanity.com/paper/2302.12931/) (2023)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning](https://mlsanity.com/paper/2109.08665/) (2021)
- [Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis?](https://mlsanity.com/paper/2403.06092/) (2024)

## Built on by (most cited first, 2 of 2)

- [Cooperative Perception: A Resource-Efficient Framework for Multi-Drone 3D Scene Reconstruction Using Federated Diffusion and NeRF](https://mlsanity.com/paper/2508.00967/) (2025)
- [A Distributed Gaussian Process Model for Multi-Robot Mapping](https://mlsanity.com/paper/2603.07351/) (2026)

## Abstract (arXiv)

Effective environment perception is crucial for enabling downstream robotic applications. Individual robotic agents often face occlusion and limited visibility issues, whereas multi-agent systems can offer a more comprehensive mapping of the environment, quicker coverage, and increased fault tolerance. In this paper, we propose a collaborative multi-agent perception system where agents collectively learn a neural radiance field (NeRF) from posed RGB images to represent a scene. Each agent processes its local sensory data and shares only its learned NeRF model with other agents, reducing communication overhead. Given NeRF's low memory footprint, this approach is well-suited for robotic systems with limited bandwidth, where transmitting all raw data is impractical. Our distributed learning framework ensures consistency across agents' local NeRF models, enabling convergence to a unified scene representation. We show the effectiveness of our method through an extensive set of experiments on datasets containing challenging real-world scenes, achieving performance comparable to centralized mapping of the environment where data is sent to a central server for processing. Additionally, we find that multi-agent learning provides regularization benefits, improving geometric consistency in scenarios with sparse input views. We show that in such scenarios, multi-agent mapping can even outperform centralized training.