# Probabilistic design of optimal sequential decision-making algorithms in learning and control

arXiv: https://arxiv.org/abs/2201.05212 · HTML: https://mlsanity.com/paper/2201.05212/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2022.

## Built on by (most cited first, 6 of 6)

- [Maximum entropy optimal density control of discrete-time linear systems and Schr"odinger bridges](https://mlsanity.com/paper/2204.05263/) (2022)
- [Distributionally Robust Free Energy Principle for Decision-Making](https://mlsanity.com/paper/2503.13223/) (2025)
- [CT-DQN: Control-Tutored Deep Reinforcement Learning](https://mlsanity.com/paper/2212.01343/) (2022)
- [An Energy-Based Mechanism for Compositional Behavior](https://mlsanity.com/paper/2512.04745/) (2025)
- [On Convex Data-Driven Inverse Optimal Control for Nonlinear, Non-stationary and Stochastic Systems](https://mlsanity.com/paper/2306.13928/) (2023)
- [DR-PETS: Learning-Based Control With Planning in Adversarial Environments](https://mlsanity.com/paper/2503.20660/) (2025)

## Abstract (arXiv)

This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.