# Meta Learning in Decentralized Neural Networks: Towards More General AI

arXiv: https://arxiv.org/abs/2302.01020 · HTML: https://mlsanity.com/paper/2302.01020/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [The Consciousness Prior](https://mlsanity.com/paper/1709.08568/) (2017)
- [Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness](https://mlsanity.com/paper/2108.03980/) (2021)
- [Homogeneous Learning: Self-Attention Decentralized Deep Learning](https://mlsanity.com/paper/2110.05290/) (2021)
- [Bidirectional Contrastive Split Learning for Visual Question Answering](https://mlsanity.com/paper/2208.11435/) (2022)

## Built on by (most cited first, 1 of 1)

- [Rethinking Meta-Learning from a Learning Lens](https://mlsanity.com/paper/2409.08474/) (2024)

## Abstract (arXiv)

Meta-learning usually refers to a learning algorithm that learns from other learning algorithms. The problem of uncertainty in the predictions of neural networks shows that the world is only partially predictable and a learned neural network cannot generalize to its ever-changing surrounding environments. Therefore, the question is how a predictive model can represent multiple predictions simultaneously. We aim to provide a fundamental understanding of learning to learn in the contents of Decentralized Neural Networks (Decentralized NNs) and we believe this is one of the most important questions and prerequisites to building an autonomous intelligence machine. To this end, we shall demonstrate several pieces of evidence for tackling the problems above with Meta Learning in Decentralized NNs. In particular, we will present three different approaches to building such a decentralized learning system: (1) learning from many replica neural networks, (2) building the hierarchy of neural networks for different functions, and (3) leveraging different modality experts to learn cross-modal representations.