# Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms

arXiv: https://arxiv.org/abs/2305.16148 · HTML: https://mlsanity.com/paper/2305.16148/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 49 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 11 of 11)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Differentiable Quality Diversity](https://mlsanity.com/paper/2106.03894/) (2021)
- [Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation](https://mlsanity.com/paper/2202.02440/) (2022)
- [Unsupervised Behaviour Discovery with Quality-Diversity Optimisation](https://mlsanity.com/paper/2106.05648/) (2021)
- [Transforming Exploratory Creativity with DeLeNoX](https://mlsanity.com/paper/2103.11715/) (2021)
- [Quality Diversity Through Surprise](https://mlsanity.com/paper/1807.02397/) (2018)
- [Generic Behaviour Similarity Measures for Evolutionary Swarm Robotics](https://mlsanity.com/paper/1304.3393/) (2013)
- [Systematic Derivation of Behaviour Characterisations in Evolutionary Robotics](https://mlsanity.com/paper/1407.0577/) (2014)

## Built on by (most cited first, 4 of 4)

- [Discovery and Deployment of Emergent Robot Swarm Behaviors via Representation Learning and Real2Sim2Real Transfer](https://mlsanity.com/paper/2502.15937/) (2025)
- [Exploring Behavior Discovery Methods for Heterogeneous Swarms of Limited-Capability Robots](https://mlsanity.com/paper/2310.16941/) (2023)
- [Representation Alignment from Human Feedback for Cross-Embodiment Reward Learning from Mixed-Quality Demonstrations](https://mlsanity.com/paper/2408.05610/) (2024)
- [SyncSBC: Decentralized Swarm Behavior Prediction for Synchronized Autonomous Control](https://mlsanity.com/paper/2608.06587/) (2026)

## Abstract (arXiv)

Robot swarms often exhibit emergent behaviors that are fascinating to observe; however, it is often difficult to predict what swarm behaviors can emerge under a given set of agent capabilities. We seek to efficiently leverage human input to automatically discover a taxonomy of collective behaviors that can emerge from a particular multi-agent system, without requiring the human to know beforehand what behaviors are interesting or even possible. Our proposed approach adapts to user preferences by learning a similarity space over swarm collective behaviors using self-supervised learning and human-in-the-loop queries. We combine our learned similarity metric with novelty search and clustering to explore and categorize the space of possible swarm behaviors. We also propose several general-purpose heuristics that improve the efficiency of our novelty search by prioritizing robot controllers that are likely to lead to interesting emergent behaviors. We test our approach in simulation on two robot capability models and show that our methods consistently discover a richer set of emergent behaviors than prior work. Code, videos, and datasets are available at https://sites.google.com/view/evolving-novel-swarms.