Data to 14 Oct 2026

Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives

arXiv 2110.15360 ↗First posted 28 Oct 2021cs.LG cs.AI cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Despite the potential of reinforcement learning (RL) for building general-purpose robotic systems, training RL agents to solve robotics tasks still remains challenging due to the difficulty of exploration in purely continuous action spaces. Addressing this problem is an active area of research with the majority of focus on improving RL methods via better optimization or more efficient exploration. An alternate but important component to consider improving is the interface of the RL algorithm with the robot. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. These parameterized primitives are expressive, simple to implement, enable efficient exploration and can be transferred across robots, tasks and environments. We perform a thorough empirical study across challenging tasks in three distinct domains with image input and a sparse terminal reward. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance irrespective of the underlying RL algorithm, significantly outperforming prior methods which learn skills from offline expert data. Code and videos at https://mihdalal.github.io/raps/

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 42 of its 64 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

42 of 64 references.

References · 42

2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
2020Mastering Atari with Discrete World Modelscited by 1,007
2015End-to-End Training of Deep Visuomotor Policiescited by 875
2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
2018Learning Dexterous In-Hand Manipulationcited by 687
2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
2020Reinforcement Learning with Augmented Datacited by 468
2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
2016The Option-Critic Architecturecited by 448

30 more

2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
2020Planning to Explore via Self-Supervised World Modelscited by 368
2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
2019Learning Latent Plans from Playcited by 316
2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 138
2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
2018Variational Option Discovery Algorithmscited by 122
2017Meta Learning Shared Hierarchiescited by 93
2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
2015Deep Reinforcement Learning in Parameterized Action Spacecited by 68
2019Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Taskscited by 67
2015Reinforcement Learning with Parameterized Actionscited by 63
2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 58
2020Neural Dynamic Policies for End-to-End Sensorimotor Learningcited by 58
2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 54
2020Learning Robot Skills with Temporal Variational Inferencecited by 54
2019Sub-policy Adaptation for Hierarchical Reinforcement Learningcited by 47
2019Efficient Bimanual Manipulation Using Learned Task Schemascited by 45
2019Dynamics-aware Embeddingscited by 34
2021LASER: Learning a Latent Action Space for Efficient Reinforcement Learningcited by 32
2020A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objectscited by 23
2021SKID RAW: Skill Discovery from Raw Trajectoriescited by 22
2020Latent Skill Planning for Exploration and Transfercited by 19
2020Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulationcited by 17
2019Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spacescited by 16
2018Hierarchical Approaches for Reinforcement Learning in Parameterized Action Spacecited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 18; 2024: 22; 2025: 8; 2026: 6.

65 papers.

Sort:

By area:

2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
2023Structured World Models from Human Videoscited by 147
2025Reinforcement Learning with Action Chunkingcited by 107
2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
2022Skill-based Model-based Reinforcement Learningcited by 50
2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 50
2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 50
2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39
2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 34
2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26

38 more

2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 26
2024Practice Makes Perfect: Planning to Learn Skill Parameter Policiescited by 23
2022Robot Learning of Mobile Manipulation with Reachability Behavior Priorscited by 22
2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 22
2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 18
2022STAP: Sequencing Task-Agnostic Policiescited by 18
2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
2024STEER: Flexible Robotic Manipulation via Dense Language Groundingcited by 14
2025Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulationcited by 14
2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skillscited by 12
2023Towards Generalist Robots: A Promising Paradigm via Generative Simulationcited by 9
2023MP3: Movement Primitive-Based (Re-)Planning Policycited by 8
2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 8
2024PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learningcited by 8
2022Meta-Learning Parameterized Skillscited by 7
2023Learning Sequential Acquisition Policies for Robot-Assisted Feedingcited by 6
2024HACMan++: Spatially-Grounded Motion Primitives for Manipulationcited by 6
2023One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentationcited by 5
2023Learning Extrinsic Dexterity with Parameterized Manipulation Primitivescited by 5
2024Redundancy-aware Action Spaces for Robot Learningcited by 5
2024Information-driven Affordance Discovery for Efficient Robotic Manipulationcited by 4
2024Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Datacited by 4
2026Learning Diffusion Policy from Primitive Skills for Robot Manipulationcited by 4
2022Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Taskscited by 3
2023PEAR: Primitive Enabled Adaptive Relabeling for Boosting Hierarchical Reinforcement Learningcited by 3
2024PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabelingcited by 3
2024SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Taskscited by 3
2025Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Taskscited by 3
2025SHaRe-RL: Structured, Interactive Reinforcement Learning for Contact-Rich Industrial Assembly Taskscited by 3
2022Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templatescited by 2
2023Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulationcited by 2
2023Efficient Learning of High Level Plans from Playcited by 2
2023CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learningcited by 2
2024Training Fast Robot Policies with Slow Foundation Modelscited by 2
2024Reinforcement Learning via Auxiliary Task Distillationcited by 2
2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learningcited by 2

50 most cited of 65 All 65 (JSON)