# Trajectory-Regularized Stochastic Optimal Control via KL Divergence

arXiv: https://arxiv.org/abs/2607.22201 · HTML: https://mlsanity.com/paper/2607.22201/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Builds on (most cited first, 5 of 5)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [A survey of the Schr"odinger problem and some of its connections with optimal transport](https://mlsanity.com/paper/1308.0215/) (2013)
- [Finite Memory Belief Approximation for Optimal Control in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2601.03132/) (2026)
- [WOMBET: World Model-Based Experience Transfer for Robust and Sample-efficient Reinforcement Learning](https://mlsanity.com/paper/2604.08958/) (2026)
- [Robust Adversarial Policy Optimization Under Dynamics Uncertainty](https://mlsanity.com/paper/2604.10974/) (2026)

## Built on by (most cited first, 1 of 1)

- [In-Distribution Imagination for Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2609.38673/) (2026)

## Abstract (arXiv)

We introduce trajectory-regularized stochastic optimal control (TRSOC), which augments standard stochastic optimal control (SOC) with a Kullback--Leibler (KL) divergence between controlled and reference trajectory distributions. Using Girsanov's theorem, the trajectory KL reduces to a quadratic drift mismatch penalty, yielding a modified running cost that preserves the dynamic programming (DP) structure. We derive the corresponding Hamilton--Jacobi--Bellman (HJB) equation and characterize the optimal policy. In the linear-quadratic (LQ) setting, the formulation admits a closed-form solution with an augmented control cost. Experiments show that the regularization parameter induces a trade-off between performance-driven and reference-preserving behavior, including cases with reference dynamics learned from offline data.