# Ergodic Control and Controlled Diffusion for Robot Learning: Review and Tutorial

arXiv: https://arxiv.org/abs/2609.13295 · HTML: https://mlsanity.com/paper/2609.13295/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 207 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Flow Matching Policy Gradients](https://mlsanity.com/paper/2507.21053/) (2025)
- [Digitizing Touch with an Artificial Multimodal Fingertip](https://mlsanity.com/paper/2411.02479/) (2024)
- [Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answering](https://mlsanity.com/paper/2507.12846/) (2025)
- [eFlesh: Highly customizable Magnetic Touch Sensing using Cut-Cell Microstructures](https://mlsanity.com/paper/2506.09994/) (2025)
- [Embodied Active Learning of Generative Sensor-Object Models](https://mlsanity.com/paper/2410.11130/) (2024)
- [Ergodic Trajectory Optimization on Generalized Domains Using Maximum Mean Discrepancy](https://mlsanity.com/paper/2410.10599/) (2024)
- [Volumetric Ergodic Control](https://mlsanity.com/paper/2511.11533/) (2025)
- [Infinite-Horizon Ergodic Control via Kernel Mean Embeddings](https://mlsanity.com/paper/2604.01023/) (2026)
- [Asymptotically Optimal Ergodic Coverage on Generalized Motion Fields](https://mlsanity.com/paper/2605.13442/) (2026)
- [RAnGE: Reachability Analysis for Guaranteed Ergodicity](https://mlsanity.com/paper/2404.03186/) (2024)
- [Scensory: Real-Time Robotic Olfactory Perception for Joint Identification and Source Localization](https://mlsanity.com/paper/2509.19318/) (2025)
- [Search at Scale: Improving Numerical Conditioning of Ergodic Coverage Optimization for Multi-Scale Domains](https://mlsanity.com/paper/2512.05229/) (2025)
- [Manufacturing Micro-Patterned Surfaces with Multi-Robot Systems](https://mlsanity.com/paper/2603.18260/) (2026)
- [Ergodic Imitation for Adaptive Exploration around Demonstrations](https://mlsanity.com/paper/2605.13996/) (2026)
- [Damage Adaptation in Seconds for Architected Materials](https://mlsanity.com/paper/2606.17394/) (2026)

## Abstract (arXiv)

Diffusion learning leverages the statistical mechanism of diffusion processes for learning, reasoning, and inferring complex distributions from data. Recent advances in diffusion learning have been transformative, with robot learning emerging as a key opportunity area, with applications spanning perception, control, and decision-making. At the same time, the statistical mechanism of diffusion processes can be controlled to shape the temporal evolution of the state distribution underlying robot trajectories, inducing ergodic behavior in robotic systems. The frameworks of controlled diffusion and ergodic control were developed around the same time as diffusion learning, and their theories and algorithms have increasingly converged. Ergodicity induced by controlled diffusion has several significant implications for robot learning, distinct from applying diffusion learning to robotics problems: it formally enforces statistical properties required to ensure optimality of robot learning, enables non-myopic search over uncertain information landscapes for data collection, and enables behavior specification based on spatial rather than temporal characteristics of trajectories. This survey introduces the intuition behind controlled diffusion for robot learning, explores its connection to diffusion learning, presents theoretical foundations and numerical tutorials for solving controlled diffusion and ergodic control problems, and reviews applications across robotics. Finally, we discuss key challenges and future opportunities in leveraging controlled diffusion for robot learning.