mlsanityData to 7 Oct 2026

Redundancy-aware Action Spaces for Robot Learning

arXiv 2406.04144 ↗First posted 6 Jun 2024cs.RO cs.AI cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Joint space and task space control are the two dominant action modes for controlling robot arms within the robot learning literature. Actions in joint space provide precise control over the robot's pose, but tend to suffer from inefficient training; actions in task space boast data-efficient training but sacrifice the ability to perform tasks in confined spaces due to limited control over the full joint configuration. This work analyses the criteria for designing action spaces for robot manipulation and introduces ER (End-effector Redundancy), a novel action space formulation that, by addressing the redundancies present in the manipulator, aims to combine the advantages of both joint and task spaces, offering fine-grained comprehensive control with overactuated robot arms whilst achieving highly efficient robot learning. We present two implementations of ER, ERAngle (ERA) and ERJoint (ERJ), and we show that ERJ in particular demonstrates superior performance across multiple settings, especially when precise control over the robot configuration is required. We validate our results both in simulated and real robotic environments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

34 of 53 references.

References · 34

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  9. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  10. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  11. 2021Implicit Behavioral Cloningcited by 442
  12. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
22 more
  1. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  2. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  3. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  4. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
  5. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  6. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  7. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  8. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
  9. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  10. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  11. 2019Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Taskscited by 56
  12. 2019PyRep: Bringing V-REP to Deep Robot Learningcited by 43
  13. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  14. 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 37
  15. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  16. 2020Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulationcited by 17
  17. 2019A Comparison of Action Spaces for Learning Manipulation Taskscited by 16
  18. 2022Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learningcited by 15
  19. 2022Coarse-to-fine Q-attention with Tree Expansioncited by 9
  20. 2022Bingham Policy Parameterization for 3D Rotations in Reinforcement Learningcited by 7
  21. 2020Joint Space Control via Deep Reinforcement Learningcited by 4
  22. 2024Information-driven Affordance Discovery for Efficient Robotic Manipulationcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2024Effective Tuning Strategies for Generalist Robot Manipulation Policiescited by 10
  2. 2025Transferring Vision-Language-Action Models to Industry Applications: Architectures, Performance, and Challengescited by 3
  3. 2025How Well do Diffusion Policies Learn Kinematic Constraint Manifolds?cited by 3
  4. 2025Hydra: Marker-Free RGB-D Hand-Eye Calibrationcited by 1
  5. 2025Kinematics-Aware Diffusion Policy with Consistent 3D Observation and Action Space for Whole-Arm Robotic Manipulationcited by 0