# The Artificial Experimentalist: Discovery and Control of Self-Organizing Phenomena with Autotelic Reinforcement Learning

arXiv: https://arxiv.org/abs/2608.26116 · HTML: https://mlsanity.com/paper/2608.26116/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey](https://mlsanity.com/paper/2012.09830/) (2020)
- [Lenia - Biology of Artificial Life](https://mlsanity.com/paper/1812.05433/) (2018)
- [Spatial Action Maps for Mobile Manipulation](https://mlsanity.com/paper/2004.09141/) (2020)
- [Lenia and Expanded Universe](https://mlsanity.com/paper/2005.03742/) (2020)
- [Automating the Search for Artificial Life with Foundation Models](https://mlsanity.com/paper/2412.17799/) (2024)
- [Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems](https://mlsanity.com/paper/2007.01195/) (2020)
- [The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence](https://mlsanity.com/paper/2307.07522/) (2023)
- [Toward Artificial Open-Ended Evolution within Lenia using Quality-Diversity](https://mlsanity.com/paper/2406.04235/) (2024)
- [Discovering Sensorimotor Agency in Cellular Automata using Diversity Search](https://mlsanity.com/paper/2402.10236/) (2024)
- [Differentiable Logic Cellular Automata: From Game of Life to Pattern Generation](https://mlsanity.com/paper/2506.04912/) (2025)
- [Neural cellular automata: applications to biology and beyond classical AI](https://mlsanity.com/paper/2509.11131/) (2025)
- [Autoverse: An Evolvable Game Language for Learning Robust Embodied Agents](https://mlsanity.com/paper/2407.04221/) (2024)
- [Exploring Flow-Lenia Universes with a Curiosity-driven AI Scientist: Discovering Diverse Ecosystem Dynamics](https://mlsanity.com/paper/2505.15998/) (2025)
- [Flow-Lenia: Emergent evolutionary dynamics in mass conservative continuous cellular automata](https://mlsanity.com/paper/2506.08569/) (2025)
- [Cooperative control of environmental extremes by artificial intelligent agents](https://mlsanity.com/paper/2212.02395/) (2022)
- [Expedition & Expansion: Leveraging Semantic Representations for Goal-Directed Exploration in Continuous Cellular Automata](https://mlsanity.com/paper/2509.03863/) (2025)
- [Visualizing the Structure of Lenia Parameter Space](https://mlsanity.com/paper/2601.01932/) (2026)

## Abstract (arXiv)

Existing methods for exploring cellular automata and other complex systems mostly operate in open loop: they set initial conditions, execute a full simulation, and observe the outcome, without intervening during execution. We introduce a closed-loop framework based on autotelic reinforcement learning, in which an agent autonomously samples diverse goals and learns a goal-conditioned policy to intervene in a complex system through minimal, local perturbations. We instantiate this framework on Lenia, a continuous cellular automaton known for life-like self-organizing patterns, in an agentic system we call CARL, and demonstrate three capabilities. First, CARL discovers stable solitons across a wide range of Lenia update rules at a higher rate than heuristic baselines. Second, it learns to steer the movement direction of existing solitons with few interventions, showing that CARL can control self-organizing patterns, not only create them. Third, humans can use trained agents to guide solitons through maze environments in real time by specifying high-level directional commands that the agent translates into low-level interventions. Trained across diverse goals, update rules, and random initial states, the agents acquire policies that generalize zero-shot to various out-of-distribution conditions. These results suggest a path toward artificial experimentalist agents that, autonomously or with human guidance, discover and control emergent phenomena in complex systems.