# On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems

arXiv: https://arxiv.org/abs/2507.21543 · HTML: https://mlsanity.com/paper/2507.21543/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 18 of 18)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Provably Efficient Maximum Entropy Exploration](https://mlsanity.com/paper/1812.02690/) (2018)
- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [On the relation between optimal transport and Schr"odinger bridges: A stochastic control viewpoint](https://mlsanity.com/paper/1412.4430/) (2014)
- [Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon](https://mlsanity.com/paper/2011.10300/) (2020)
- [Mutual-Information Regularization in Markov Decision Processes and Actor-Critic Learning](https://mlsanity.com/paper/1909.05950/) (2019)
- [Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradient](https://mlsanity.com/paper/2007.01932/) (2020)
- [Fast Policy Learning for Linear Quadratic Control with Entropy Regularization](https://mlsanity.com/paper/2311.14168/) (2023)
- [Robust transport over networks](https://mlsanity.com/paper/1603.08129/) (2016)
- [Deep RL With Information Constrained Policies: Generalization in Continuous Control](https://mlsanity.com/paper/2010.04646/) (2020)
- [Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients](https://mlsanity.com/paper/2012.15019/) (2020)
- [Maximum entropy optimal density control of discrete-time linear systems and Schr"odinger bridges](https://mlsanity.com/paper/2204.05263/) (2022)
- [Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost](https://mlsanity.com/paper/2309.10662/) (2023)
- [Mutual Information Optimal Control of Discrete-Time Linear Systems](https://mlsanity.com/paper/2507.04712/) (2025)

## Built on by (most cited first, 1 of 1)

- [Mutual Information Optimal Density Control of Linear Systems and Generalized Schr"odinger Bridges with Reference Refinement](https://mlsanity.com/paper/2605.09349/) (2026)

## Abstract (arXiv)

Mutual information regularization has recently been studied in reinforcement learning (RL) as an extension of entropy regularization, in which not only the policy but also the reference distribution (prior) is optimized, and has also been used for privacy preserving policy design. To obtain theoretical insight into how this joint optimization affects policy stochasticity, we study a model-based linear-quadratic setting that preserves this key structure while permitting an explicit analysis. Specifically, we consider a mutual information optimal control problem (MIOCP) for stochastic discrete-time linear systems with quadratic costs and Gaussian policies and priors. We first review alternating optimization of the policy and the prior with a technical extension. We then establish the existence of an optimal solution, characterize its covariance matrices, and derive sufficient conditions under which the optimal policy is either a stochastic feedback policy or a deterministic open-loop policy. We further interpret this characterization in terms of the state information disclosed through the control input. We also analyze the limiting behavior of alternating optimization under the sufficient conditions. The validity of the theoretical results is demonstrated through numerical experiments.