# DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning

arXiv: https://arxiv.org/abs/2109.08665 · HTML: https://mlsanity.com/paper/2109.08665/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2021.

## Built on by (most cited first, 16 of 16)

- [Heterogeneous Multi-Robot Reinforcement Learning](https://mlsanity.com/paper/2301.07137/) (2023)
- [Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices](https://mlsanity.com/paper/2403.09141/) (2024)
- [Federated Multi-Agent Mapping for Planetary Exploration](https://mlsanity.com/paper/2404.02289/) (2024)
- [Multi-Agent Reinforcement Learning for Joint Police Patrol and Dispatch](https://mlsanity.com/paper/2409.02246/) (2024)
- [Distributed NeRF Learning for Collaborative Multi-Robot Perception](https://mlsanity.com/paper/2409.20289/) (2024)
- [Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus](https://mlsanity.com/paper/2306.13892/) (2023)
- [Distributed Variational Inference for Online Supervised Learning](https://mlsanity.com/paper/2309.02606/) (2023)
- [Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning](https://mlsanity.com/paper/2403.06535/) (2024)
- [Edge AI Collaborative Learning: Bayesian Approaches to Uncertainty Estimation](https://mlsanity.com/paper/2410.08651/) (2024)
- [Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systems](https://mlsanity.com/paper/2506.05577/) (2025)
- [Connected Autonomous Vehicle Motion Planning with Video Predictions from Smart, Self-Supervised Infrastructure](https://mlsanity.com/paper/2309.07504/) (2023)
- [Uncertainty Estimation in Multi-Agent Distributed Learning](https://mlsanity.com/paper/2311.13356/) (2023)
- [Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers](https://mlsanity.com/paper/2401.15838/) (2024)
- [Personalized and Resilient Distributed Learning Through Opinion Dynamics](https://mlsanity.com/paper/2505.14081/) (2025)
- [CoCoL: A Communication Efficient Decentralized Collaborative Method for Multi-Robot Systems](https://mlsanity.com/paper/2508.20898/) (2025)
- [A Distributed Gaussian Process Model for Multi-Robot Mapping](https://mlsanity.com/paper/2603.07351/) (2026)

## Abstract (arXiv)

We present a distributed algorithm that enables a group of robots to collaboratively optimize the parameters of a deep neural network model while communicating over a mesh network. Each robot only has access to its own data and maintains its own version of the neural network, but eventually learns a model that is as good as if it had been trained on all the data centrally. No robot sends raw data over the wireless network, preserving data privacy and ensuring efficient use of wireless bandwidth. At each iteration, each robot approximately optimizes an augmented Lagrangian function, then communicates the resulting weights to its neighbors, updates dual variables, and repeats. Eventually, all robots' local network weights reach a consensus. For convex objective functions, we prove this consensus is a global optimum. We compare our algorithm to two existing distributed deep neural network training algorithms in (i) an MNIST image classification task, (ii) a multi-robot implicit mapping task, and (iii) a multi-robot reinforcement learning task. In all of our experiments our method out performed baselines, and was able to achieve validation loss equivalent to centrally trained models. See hrefhttps://msl.stanford.edu/projects/distnntrainhttps://msl.stanford.edu/projects/distnntrain for videos and a link to our GitHub repository.