# Collective Learning by Ensembles of Altruistic Diversifying Neural Networks

arXiv: https://arxiv.org/abs/2006.11671 · HTML: https://mlsanity.com/paper/2006.11671/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2020.

## Built on by (most cited first, 3 of 3)

- [lo-fi: distributed fine-tuning without communication](https://mlsanity.com/paper/2210.11948/) (2022)
- [Decentralized Low-Latency Collaborative Inference via Ensembles on the Edge](https://mlsanity.com/paper/2206.03165/) (2022)
- [Semantic representations emerge in biologically inspired ensembles of cross-supervising neural networks](https://mlsanity.com/paper/2510.14486/) (2025)

## Abstract (arXiv)

Combining the predictions of collections of neural networks often outperforms the best single network. Such ensembles are typically trained independently, and their superior `wisdom of the crowd' originates from the differences between networks. Collective foraging and decision making in socially interacting animal groups is often improved or even optimal thanks to local information sharing between conspecifics. We therefore present a model for co-learning by ensembles of interacting neural networks that aim to maximize their own performance but also their functional relations to other networks. We show that ensembles of interacting networks outperform independent ones, and that optimal ensemble performance is reached when the coupling between networks increases diversity and degrades the performance of individual networks. Thus, even without a global goal for the ensemble, optimal collective behavior emerges from local interactions between networks. We show the scaling of optimal coupling strength with ensemble size, and that networks in these ensembles specialize functionally and become more `confident' in their assessments. Moreover, optimal co-learning networks differ structurally, relying on sparser activity, a wider range of synaptic weights, and higher firing rates - compared to independently trained networks. Finally, we explore interactions-based co-learning as a framework for expanding and boosting ensembles.