# Velocity-History-Based Soft Actor-Critic Tackling IROS'24 Competition "AI Olympics with RealAIGym"

arXiv: https://arxiv.org/abs/2410.20096 · HTML: https://mlsanity.com/paper/2410.20096/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 14 references that are in the map.
- First posted on arXiv 26 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 5 of 5)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Assessing Transferability from Simulation to Reality for Reinforcement Learning](https://mlsanity.com/paper/1907.04685/) (2019)
- [Solving the swing-up and balance task for the Acrobot and Pendubot with SAC](https://mlsanity.com/paper/2312.11311/) (2023)

## Built on by (most cited first, 1 of 1)

- [Average-Reward Maximum Entropy Reinforcement Learning for Global Policy in Double Pendulum Tasks](https://mlsanity.com/paper/2505.07516/) (2025)

## Abstract (arXiv)

The ``AI Olympics with RealAIGym'' competition challenges participants to stabilize chaotic underactuated dynamical systems with advanced control algorithms. In this paper, we present a novel solution submitted to IROS'24 competition, which builds upon Soft Actor-Critic (SAC), a popular model-free entropy-regularized Reinforcement Learning (RL) algorithm. We add a `context' vector to the state, which encodes the immediate history via a Convolutional Neural Network (CNN) to counteract the unmodeled effects on the real system. Our method achieves high performance scores and competitive robustness scores on both tracks of the competition: Pendubot and Acrobot.