# Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach

arXiv: https://arxiv.org/abs/2603.11757 · HTML: https://mlsanity.com/paper/2603.11757/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 15 of 15)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [Distributed Cooperative Decision-Making in Multiarmed Bandits: Frequentist and Bayesian Algorithms](https://mlsanity.com/paper/1606.00911/) (2016)
- [Emergent Social Learning via Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2010.00581/) (2020)
- [PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning](https://mlsanity.com/paper/2102.12560/) (2021)
- [A social path to human-like artificial intelligence](https://mlsanity.com/paper/2405.15815/) (2024)
- [Observational Learning by Reinforcement Learning](https://mlsanity.com/paper/1706.06617/) (2017)
- [An Information-theoretic On-line Learning Principle for Specialization in Hierarchical Decision-Making Systems](https://mlsanity.com/paper/1907.11452/) (2019)
- [Unpacking Human-AI interactions: From interaction primitives to a design space](https://mlsanity.com/paper/2401.05115/) (2024)
- [Generalized Thompson Sampling for Sequential Decision-Making and Causal Inference](https://mlsanity.com/paper/1303.4431/) (2013)
- [Meta-control of social learning strategies](https://mlsanity.com/paper/2106.10015/) (2021)
- [Bounded rational decision-making from elementary computations that reduce uncertainty](https://mlsanity.com/paper/1904.03964/) (2019)
- [Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning](https://mlsanity.com/paper/2204.12371/) (2022)
- [Flexible social inference facilitates targeted social learning when rewards are not observable](https://mlsanity.com/paper/2212.00869/) (2022)

## Abstract (arXiv)

Personalized AI-based services involve a population of individual reinforcement learning agents. However, most reinforcement learning algorithms focus on harnessing individual learning and fail to leverage the social learning capabilities commonly exhibited by humans and animals. Social learning integrates individual experience with observing others' behavior, presenting opportunities for improved learning outcomes. In this study, we focus on a social bandit learning scenario where a social agent observes other agents' actions without knowledge of their rewards. The agents independently pursue their own policy without explicit motivation to teach each other. We propose a free energy-based social bandit learning algorithm over the policy space, where the social agent evaluates others' expertise levels without resorting to any oracle or social norms. Accordingly, the social agent integrates its direct experiences in the environment and others' estimated policies. The theoretical convergence of our algorithm to the optimal policy is proven. Empirical evaluations validate the superiority of our social learning method over alternative approaches in various scenarios. Our algorithm strategically identifies the relevant agents, even in the presence of random or suboptimal agents, and skillfully exploits their behavioral information. In addition to societies including expert agents, in the presence of relevant but non-expert agents, our algorithm significantly enhances individual learning performance, where most related methods fail. Importantly, it also maintains logarithmic regret.