mlsanityData to 7 Oct 2026

Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks

arXiv 2110.03655 ↗First posted 7 Oct 2021cs.LG cs.AI cs.ROcited by 75

Not on the map. Why?

Abstract (arXiv)

Realistic manipulation tasks require a robot to interact with an environment with a prolonged sequence of motor actions. While deep reinforcement learning methods have recently emerged as a promising paradigm for automating manipulation behaviors, they usually fall short in long-horizon tasks due to the exploration burden. This work introduces Manipulation Primitive-augmented reinforcement Learning (MAPLE), a learning framework that augments standard reinforcement learning algorithms with a pre-defined library of behavior primitives. These behavior primitives are robust functional modules specialized in achieving manipulation goals, such as grasping and pushing. To use these heterogeneous primitives, we develop a hierarchical policy that involves the primitives and instantiates their executions with input parameters. We demonstrate that MAPLE outperforms baseline approaches by a significant margin on a suite of simulated manipulation tasks. We also quantify the compositional structure of the learned behaviors and highlight our method's ability to transfer policies to new task variants and to physical hardware. Videos and code are available at https://ut-austin-rpl.github.io/maple

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 59 of its 73 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

59 of 73 references.

References · 59

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  10. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  11. 2019Solving Rubik's Cube with a Robot Handcited by 699
  12. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
38 more
  1. 2018Learning Dexterous In-Hand Manipulationcited by 643
  2. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  6. 2016The Option-Critic Architecturecited by 433
  7. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  10. 2020Integrated Task and Motion Planningcited by 281
  11. 2019Self-Supervised Exploration via Disagreementcited by 263
  12. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  13. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  14. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  15. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  16. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  17. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  18. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  19. 2019Planning with Goal-Conditioned Policiescited by 144
  20. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  21. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  22. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  23. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  24. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  25. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 80
  26. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 75
  27. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 65
  28. 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
  29. 2015Reinforcement Learning with Parameterized Actionscited by 61
  30. 2018Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstrationcited by 60
  31. 2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 54
  32. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 53
  33. 2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
  34. 2020Learning compositional models of robot skills for task and motion planningcited by 47
  35. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  36. 2019Efficient Bimanual Manipulation Using Learned Task Schemascited by 44
  37. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  38. 2020Continuous-Discrete Reinforcement Learning for Hybrid Control in Roboticscited by 42

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 18; 2024: 26; 2025: 15; 2026: 6.

75 papers.

Sort:

By area:

  1. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  3. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 87
  4. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
  5. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  6. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  7. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  8. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  9. 2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39
  10. 2024VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planningcited by 37
  11. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  12. 2023MAVERIC: A Data-Driven Approach to Personalized Autonomous Drivingcited by 22
38 more
  1. 2024Practice Makes Perfect: Planning to Learn Skill Parameter Policiescited by 22
  2. 2022STAP: Sequencing Task-Agnostic Policiescited by 17
  3. 2023Primitive Skill-based Robot Learning from Human Evaluative Feedbackcited by 15
  4. 2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
  5. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  6. 2024STEER: Flexible Robotic Manipulation via Dense Language Groundingcited by 13
  7. 2025Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulationcited by 13
  8. 2023Efficient Planning with Latent Diffusioncited by 11
  9. 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushingcited by 9
  10. 2023NOIR: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 8
  11. 2024PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learningcited by 8
  12. 2024RecoveryChaining: Learning Local Recovery Policies for Robust Manipulationcited by 8
  13. 2025ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planningcited by 8
  14. 2022Meta-Learning Parameterized Skillscited by 7
  15. 2022Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environmentscited by 7
  16. 2023A Closer Look at Reward Decomposition for High-Level Robotic Explanationscited by 7
  17. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  18. 2024Diffusion-Reinforcement Learning Hierarchical Motion Planning in Multi-agent Adversarial Gamescited by 7
  19. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewardscited by 7
  20. 2022E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidancecited by 6
  21. 2023Learning Sequential Acquisition Policies for Robot-Assisted Feedingcited by 6
  22. 2024HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environmentscited by 6
  23. 2024Redundancy-aware Action Spaces for Robot Learningcited by 5
  24. 2024HACMan++: Spatially-Grounded Motion Primitives for Manipulationcited by 5
  25. 2022Sampling Through the Lens of Sequential Decision Makingcited by 4
  26. 2023One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentationcited by 4
  27. 2024Deployable Reinforcement Learning with Variable Control Ratecited by 4
  28. 2024Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Datacited by 4
  29. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
  30. 2022Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Taskscited by 3
  31. 2023Learning Extrinsic Dexterity with Parameterized Manipulation Primitivescited by 3
  32. 2024PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabelingcited by 3
  33. 2024Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulationcited by 3
  34. 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 3
  35. 2025SLAP: Shortcut Learning for Abstract Planningcited by 3
  36. 2022Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templatescited by 2
  37. 2023Efficient Learning of High Level Plans from Playcited by 2
  38. 2023CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learningcited by 2

50 most cited of 75 All 75 (JSON)