# Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

arXiv: https://arxiv.org/abs/2605.15938 · HTML: https://mlsanity.com/paper/2605.15938/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 48 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 3 of 3)

- [Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark](https://mlsanity.com/paper/2302.00706/) (2023)
- [Q-learning with temporal memory to navigate turbulence](https://mlsanity.com/paper/2404.17495/) (2024)
- [Optimal policies for Bayesian olfactory search in turbulent flows](https://mlsanity.com/paper/2207.04277/) (2022)

## Abstract (arXiv)

Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.