mlsanityData to 13 Oct 2026

Learning Design and Construction with Varying-Sized Materials via Prioritized Memory Resets

arXiv 2204.05509 ↗First posted 12 Apr 2022cs.RO cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Can a robot autonomously learn to design and construct a bridge from varying-sized blocks without a blueprint? It is a challenging task with long horizon and sparse reward -- the robot has to figure out physically stable design schemes and feasible actions to manipulate and transport blocks. Due to diverse block sizes, the state space and action trajectories are vast to explore. In this paper, we propose a hierarchical approach for this problem. It consists of a reinforcement-learning designer to propose high-level building instructions and a motion-planning-based action generator to manipulate blocks at the low level. For high-level learning, we develop a novel technique, prioritized memory resetting (PMR) to improve exploration. PMR adaptively resets the state to those most critical configurations from a replay buffer so that the robot can resume training on partial architectures instead of from scratch. Furthermore, we augment PMR with auxiliary training objectives and fine-tune the designer with the locomotion generator. Our experiments in simulation and on a real deployed robotic system demonstrate that it is able to effectively construct bridges with blocks of varying sizes at a high success rate. Demos can be found at https://sites.google.com/view/bridge-pmr.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

33 of 47 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,106
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,334
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,180
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,175
  5. 2018Exploration by Random Network Distillationcited by 971
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 731
  7. 2019Solving Rubik's Cube with a Robot Handcited by 702
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 687
  9. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 442
  10. 2016The Option-Critic Architecturecited by 439
  11. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  12. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 325
21 more
  1. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  2. 2018Large-Scale Study of Curiosity-Driven Learningcited by 279
  3. 2020First return, then explorecited by 257
  4. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 254
  5. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 186
  6. 2020Rearrangement: A Challenge for Embodied AIcited by 154
  7. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 140
  8. 2020Phasic Policy Gradientcited by 123
  9. 2020Self-Supervised Policy Adaptation during Deploymentcited by 122
  10. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 108
  11. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  12. 2020Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphscited by 83
  13. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  14. 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
  15. 2016Learning and Transfer of Modulated Locomotor Controllerscited by 59
  16. 2019Towards Practical Multi-Object Manipulation using Relational Reinforcement Learningcited by 54
  17. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 52
  18. 2019Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassemblycited by 51
  19. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  20. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 33
  21. 2021Learning to Design and Construct Bridge without Blueprintcited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learningcited by 1
  2. 2026Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Planscited by 1