# Average-Reward Maximum Entropy Reinforcement Learning for Underactuated Double Pendulum Tasks

arXiv: https://arxiv.org/abs/2409.08938 · HTML: https://mlsanity.com/paper/2409.08938/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [Average-reward model-free reinforcement learning: a systematic review and literature mapping](https://mlsanity.com/paper/2010.08920/) (2020)
- [Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints](https://mlsanity.com/paper/1807.03223/) (2018)
- [Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation](https://mlsanity.com/paper/2407.18143/) (2024)
- [Solving the swing-up and balance task for the Acrobot and Pendubot with SAC](https://mlsanity.com/paper/2312.11311/) (2023)

## Built on by (most cited first, 1 of 1)

- [Average-Reward Maximum Entropy Reinforcement Learning for Global Policy in Double Pendulum Tasks](https://mlsanity.com/paper/2505.07516/) (2025)

## Abstract (arXiv)

This report presents a solution for the swing-up and stabilisation tasks of the acrobot and the pendubot, developed for the AI Olympics competition at IROS 2024. Our approach employs the Average-Reward Entropy Advantage Policy Optimization (AR-EAPO), a model-free reinforcement learning (RL) algorithm that combines average-reward RL and maximum entropy RL. Results demonstrate that our controller achieves improved performance and robustness scores compared to established baseline methods in both the acrobot and pendubot scenarios, without the need for a heavily engineered reward function or system model. The current results are applicable exclusively to the simulation stage setup.