# Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning

arXiv: https://arxiv.org/abs/2204.12371 · HTML: https://mlsanity.com/paper/2204.12371/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 62 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Path Planning using Neural A* Search](https://mlsanity.com/paper/2009.07476/) (2020)
- [Social Learning in Multi Agent Multi Armed Bandits](https://mlsanity.com/paper/1910.02100/) (2019)
- [Emergent Social Learning via Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2010.00581/) (2020)
- [Stacking Models for Nearly Optimal Link Prediction in Complex Networks](https://mlsanity.com/paper/1909.07578/) (2019)
- [A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings](https://mlsanity.com/paper/2106.09012/) (2021)
- [Robust Multi-Agent Multi-Armed Bandits](https://mlsanity.com/paper/2007.03812/) (2020)
- [Social learning strategies modify the effect of network structure on group performance](https://mlsanity.com/paper/1606.00753/) (2016)
- [Neural-Network Heuristics for Adaptive Bayesian Quantum Estimation](https://mlsanity.com/paper/2003.02183/) (2020)
- [Meta-control of social learning strategies](https://mlsanity.com/paper/2106.10015/) (2021)

## Built on by (most cited first, 3 of 3)

- [Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning](https://mlsanity.com/paper/2406.00392/) (2024)
- [Generative AI collective behavior needs an interactionist paradigm](https://mlsanity.com/paper/2601.10567/) (2026)
- [Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach](https://mlsanity.com/paper/2603.11757/) (2026)

## Abstract (arXiv)

How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.