# Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics

arXiv: https://arxiv.org/abs/2506.00138 · HTML: https://mlsanity.com/paper/2506.00138/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 71 references that are in the map.
- First posted on arXiv 30 May 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Discovering and Achieving Goals via World Models](https://mlsanity.com/paper/2110.09514/) (2021)
- [Generalized Shape Metrics on Neural Representations](https://mlsanity.com/paper/2110.14739/) (2021)
- [Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning](https://mlsanity.com/paper/1703.01732/) (2017)
- [Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution](https://mlsanity.com/paper/2210.08340/) (2022)
- [Learning to Play with Intrinsically-Motivated Self-Aware Agents](https://mlsanity.com/paper/1802.07442/) (2018)
- [Active World Model Learning with Progress Curiosity](https://mlsanity.com/paper/2007.07853/) (2020)
- [Deep neuroethology of a virtual rodent](https://mlsanity.com/paper/1911.09451/) (2019)
- [Curiosity-Driven Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2501.11463/) (2025)
- [Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes](https://mlsanity.com/paper/2305.11772/) (2023)
- [Brain-Model Evaluations Need the NeuroAI Turing Test](https://mlsanity.com/paper/2502.16238/) (2025)
- [Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity](https://mlsanity.com/paper/2111.01760/) (2021)
- [Neuron-Astrocyte Associative Memory](https://mlsanity.com/paper/2311.08135/) (2023)
- [Contextual guidance: An integrated theory for astrocytes function in brain circuits and behavior](https://mlsanity.com/paper/2211.09906/) (2022)
- [The Causal Role of Astrocytes in Slow-Wave Rhythmogenesis: A Computational Modelling Study](https://mlsanity.com/paper/1702.03993/) (2017)
- [Astrocytes as a mechanism for meta-plasticity and contextually-guided network function](https://mlsanity.com/paper/2311.03508/) (2023)

## Built on by (most cited first, 3 of 3)

- [Whole-Brain Connectomic Graph Model Enables Whole-Body Locomotion Control in Fruit Fly](https://mlsanity.com/paper/2602.17997/) (2026)
- [What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty](https://mlsanity.com/paper/2603.02491/) (2026)
- [Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents](https://mlsanity.com/paper/2510.03699/) (2025)

## Abstract (arXiv)

Autonomy is a hallmark of animal intelligence, enabling adaptive and intelligent behavior in complex environments without relying on external reward or task structure. Existing reinforcement learning approaches to exploration in reward-free environments, including a class of methods known as model-based intrinsic motivation, exhibit inconsistent exploration patterns and do not converge to an exploratory policy, thus failing to capture robust autonomous behaviors observed in animals. Moreover, systems neuroscience has largely overlooked the neural basis of autonomy, focusing instead on experimental paradigms where animals are motivated by external reward rather than engaging in ethological, naturalistic and task-independent behavior. To bridge these gaps, we introduce a novel model-based intrinsic drive explicitly designed after the principles of autonomous exploration in animals. Our method (3M-Progress) achieves animal-like exploration by tracking divergence between an online world model and a fixed prior learned from an ecological niche. To the best of our knowledge, we introduce the first autonomous embodied agent that predicts brain data entirely from self-supervised optimization of an intrinsic goal -- without any behavioral or neural training data -- demonstrating that 3M-Progress agents capture the explainable variance in behavioral patterns and whole-brain neural-glial dynamics recorded from autonomously behaving larval zebrafish, thereby providing the first goal-driven, population-level model of neural-glial computation. Our findings establish a computational framework connecting model-based intrinsic motivation to naturalistic behavior, providing a foundation for building artificial agents with animal-like autonomy.