# Universal Agent Mixtures and the Geometry of Intelligence

arXiv: https://arxiv.org/abs/2302.06083 · HTML: https://mlsanity.com/paper/2302.06083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Builds on (most cited first, 7 of 7)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agents](https://mlsanity.com/paper/2102.08159/) (2021)
- [Bad Universal Priors and Notions of Optimality](https://mlsanity.com/paper/1510.04931/) (2015)
- [Reward-Punishment Symmetric Universal Intelligence](https://mlsanity.com/paper/2110.02450/) (2021)
- [Feudal Steering: Hierarchical Learning for Steering Angle Prediction](https://mlsanity.com/paper/2006.06869/) (2020)

## Abstract (arXiv)

Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents' expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performance across environments, the weighted mixture's intelligence is the weighted average of the original agents' intelligences. This operation enables various interesting new theorems that shed light on the geometry of RL agent intelligence, namely: results about symmetries, convex agent-sets, and local extrema. We also show that any RL agent intelligence measure based on average performance across environments, subject to certain weak technical conditions, is identical (up to a constant factor) to performance within a single environment dependent on said intelligence measure.