# Tsallis Entropy Regularization for Linear Quadratic Regulator and Kullback-Leibler Control

arXiv: https://arxiv.org/abs/2403.01805 · HTML: https://mlsanity.com/paper/2403.01805/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [A unified view of entropy-regularized Markov decision processes](https://mlsanity.com/paper/1705.07798/) (2017)
- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [Sparse Markov Decision Processes with Causal Sparse Tsallis Entropy Regularization for Reinforcement Learning](https://mlsanity.com/paper/1709.06293/) (2017)
- [Effective Exploration for Deep Reinforcement Learning via Bootstrapped Q-Ensembles under Tsallis Entropy Regularization](https://mlsanity.com/paper/1809.00403/) (2018)
- [Maximum entropy optimal density control of discrete-time linear systems and Schr"odinger bridges](https://mlsanity.com/paper/2204.05263/) (2022)
- [Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost](https://mlsanity.com/paper/2309.10662/) (2023)
- [Sparse and Constrained Stochastic Predictive Control for Networked Systems](https://mlsanity.com/paper/1706.01792/) (2017)
- [Sparse optimal stochastic control](https://mlsanity.com/paper/2109.07716/) (2021)

## Abstract (arXiv)

Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft Actor-Critic. The aim of this paper is to show that formulations based on Tsallis entropy, which is a one-parameter extension of Shannon entropy, retain many of the structural and computational advantages of Shannon-entropy-based approaches while offering additional benefits. In particular, we derive a closed-form solution for the linear quadratic regulator and an efficient computational method for the Kullback-Leibler control problem. We also demonstrate its usefulness in balancing between exploration and sparsity of the obtained control law.