mlsanityData to 14 Oct 2026

Dynamic Entropy Tuning in Reinforcement Learning Low-Level Quadcopter Control: Stochasticity vs Determinism

arXiv 2512.18336 ↗First posted 20 Dec 2025cs.RO cs.AI cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper explores the impact of dynamic entropy tuning in Reinforcement Learning (RL) algorithms that train a stochastic policy. Its performance is compared against algorithms that train a deterministic one. Stochastic policies optimize a probability distribution over actions to maximize rewards, while deterministic policies select a single deterministic action per state. The effect of training a stochastic policy with both static entropy and dynamic entropy and then executing deterministic actions to control the quadcopter is explored. It is then compared against training a deterministic policy and executing deterministic actions. For the purpose of this research, the Soft Actor-Critic (SAC) algorithm was chosen for the stochastic algorithm while the Twin Delayed Deep Deterministic Policy Gradient (TD3) was chosen for the deterministic algorithm. The training and simulation results show the positive effect the dynamic entropy tuning has on controlling the quadcopter by preventing catastrophic forgetting and improving exploration efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 12 references that are in the map. First posted on arXiv 20 Dec 2025.

Builds on

7 of 12 references.

References · 7

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  3. 2019Better Exploration with Optimistic Actor-Criticcited by 88
  4. 2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 77
  5. 2019Adaptive Online Planning for Continual Lifelong Learningcited by 7
  6. 2020Using Soft Actor-Critic for Low-Level UAV Controlcited by 5
  7. 2021A Model-free Deep Reinforcement Learning Approach To Maneuver A Quadrotor Despite Single Rotor Failurecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Network Distributed Multi-Agent Reinforcement Learning for Consensus Control of Quadcopterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL
  2. 2025Effective Reinforcement Learning Control using Conservative Soft Actor-Critic
  3. 2025Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learning
  4. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  5. 2025Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with Multiplicative Noise
  6. 2026Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
  7. 2025When Maximum Entropy Misleads Policy Optimization
  8. 2025Max-Entropy Reinforcement Learning with Flow Matching and A Case Study on LQR