# A Comparison of Action Spaces for Learning Manipulation Tasks

arXiv: https://arxiv.org/abs/1908.08659 · HTML: https://mlsanity.com/paper/1908.08659/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [LASER: Learning a Latent Action Space for Efficient Reinforcement Learning](https://mlsanity.com/paper/2103.15793/) (2021)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer](https://mlsanity.com/paper/2312.03673/) (2023)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Learning to Centralize Dual-Arm Assembly](https://mlsanity.com/paper/2110.04003/) (2021)
- [Continual Domain Randomization](https://mlsanity.com/paper/2403.12193/) (2024)
- [CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces](https://mlsanity.com/paper/2211.15824/) (2022)
- [Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space](https://mlsanity.com/paper/2110.09904/) (2021)
- [Redundancy-aware Action Spaces for Robot Learning](https://mlsanity.com/paper/2406.04144/) (2024)
- [Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion](https://mlsanity.com/paper/2210.08065/) (2022)
- [How Well do Diffusion Policies Learn Kinematic Constraint Manifolds?](https://mlsanity.com/paper/2510.01404/) (2025)
- [End-to-End Stable Imitation Learning via Autonomous Neural Dynamic Policies](https://mlsanity.com/paper/2305.12886/) (2023)
- [Investigating the Impact of Action Representations in Policy Gradient Algorithms](https://mlsanity.com/paper/2309.06921/) (2023)
- [Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2606.18594/) (2026)
- [A Learning-based Adaptive Compliance Method for Symmetric Bi-manual Manipulation](https://mlsanity.com/paper/2303.15262/) (2023)
- [Kinematics-Aware Diffusion Policy with Consistent 3D Observation and Action Space for Whole-Arm Robotic Manipulation](https://mlsanity.com/paper/2512.17568/) (2025)

## Abstract (arXiv)

Designing reinforcement learning (RL) problems that can produce delicate and precise manipulation policies requires careful choice of the reward function, state, and action spaces. Much prior work on applying RL to manipulation tasks has defined the action space in terms of direct joint torques or reference positions for a joint-space proportional derivative (PD) controller. In practice, it is often possible to add additional structure by taking advantage of model-based controllers that support both accurate positioning and control of the dynamic response of the manipulator. In this paper, we evaluate how the choice of action space for dynamic manipulation tasks affects the sample complexity as well as the final quality of learned policies. We compare learning performance across three tasks (peg insertion, hammering, and pushing), four action spaces (torque, joint PD, inverse dynamics, and impedance control), and using two modern reinforcement learning algorithms (Proximal Policy Optimization and Soft Actor-Critic). Our results lend support to the hypothesis that learning references for a task-space impedance controller significantly reduces the number of samples needed to achieve good performance across all tasks and algorithms.