# A Q-learning approach to the continuous control problem of robot inverted pendulum balancing

arXiv: https://arxiv.org/abs/2312.02649 · HTML: https://mlsanity.com/paper/2312.02649/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Learning Contact-Rich Manipulation Skills with Guided Policy Search](https://mlsanity.com/paper/1501.05611/) (2015)
- [CAQL: Continuous Action Q-Learning](https://mlsanity.com/paper/1909.12397/) (2019)
- [RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration](https://mlsanity.com/paper/1906.07372/) (2019)
- [Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation](https://mlsanity.com/paper/1910.09470/) (2019)
- [Reinforced Grounded Action Transformation for Sim-to-Real Transfer](https://mlsanity.com/paper/2008.01279/) (2020)
- [Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control](https://mlsanity.com/paper/1712.03303/) (2017)
- [A Framework for Learning from Demonstration with Minimal Human Effort](https://mlsanity.com/paper/2306.09211/) (2023)
- [KUKA Sunrise Toolbox: Interfacing Collaborative Robots with MATLAB](https://mlsanity.com/paper/1709.01438/) (2017)
- [Learning Robust Manipulation Skills with Guided Policy Search via Generative Motor Reflexes](https://mlsanity.com/paper/1809.05714/) (2018)
- [Deep reinforcement learning applied to an assembly sequence planning problem with user preferences](https://mlsanity.com/paper/2304.06567/) (2023)

## Abstract (arXiv)

This study evaluates the application of a discrete action space reinforcement learning method (Q-learning) to the continuous control problem of robot inverted pendulum balancing. To speed up the learning process and to overcome technical difficulties related to the direct learning on the real robotic system, the learning phase is performed in simulation environment. A mathematical model of the system dynamics is implemented, deduced by curve fitting on data acquired from the real system. The proposed approach demonstrated feasible, featuring its application on a real world robot that learned to balance an inverted pendulum. This study also reinforces and demonstrates the importance of an accurate representation of the physical world in simulation to achieve a more efficient implementation of reinforcement learning algorithms in real world, even when using a discrete action space algorithm to control a continuous action.