mlsanityData to 14 Oct 2026

Robotic Test Tube Rearrangement Using Combined Reinforcement Learning and Motion Planning

arXiv 2401.09772 ↗First posted 18 Jan 2024cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

A combined task-level reinforcement learning and motion planning framework is proposed in this paper to address a multi-class in-rack test tube rearrangement problem. At the task level, the framework uses reinforcement learning to infer a sequence of swap actions while ignoring robotic motion details. At the motion level, the framework accepts the swapping action sequences inferred by task-level agents and plans the detailed robotic pick-and-place motion. The task and motion-level planning form a closed loop with the help of a condition set maintained for each rack slot, which allows the framework to perform replanning and effectively find solutions in the presence of low-level failures. Particularly for reinforcement learning, the framework leverages a distributed deep Q-learning structure with the Dueling Double Deep Q Network (D3QN) to acquire near-optimal policies and uses an A^star-based post-processing technique to amplify the collected training data. The D3QN and distributed learning help increase training efficiency. The post-processing helps complete unfinished action sequences and remove redundancy, thus making the training data more effective. We carry out both simulations and real-world studies to understand the performance of the proposed framework. The results verify the performance of the RL and post-processing and show that the closed-loop combination improves robustness. The framework is ready to incorporate various sensory feedback. The real-world studies also demonstrated the incorporation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 34 of its 60 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

34 of 60 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  8. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  9. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  10. 2018Distributed Prioritized Experience Replaycited by 269
  11. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  12. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
22 more
  1. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 106
  2. 2021Visual Room Rearrangementcited by 102
  3. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 59
  4. 2021Heuristic-Guided Reinforcement Learningcited by 42
  5. 2021NeRP: Neural Rearrangement Planning for Unknown Objectscited by 34
  6. 2019Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planningcited by 24
  7. 2021Predicting Stable Configurations for Semantic Placement of Novel Objectscited by 24
  8. 2022Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objectscited by 16
  9. 2019Multi-Object Rearrangement with Monte Carlo Tree Search:A Case Study on Planar Nonprehensile Sortingcited by 13
  10. 2022Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifierscited by 12
  11. 2017Complexity Results and Fast Methods for Optimal Tabletop Rearrangement with Overhand Graspscited by 10
  12. 2021Manipulation Planning Among Movable Obstacles Using Physics-Based Adaptive Motion Primitivescited by 8
  13. 2023Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearrangingcited by 8
  14. 2023Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planningcited by 8
  15. 2021On Minimizing the Number of Running Buffers for Tabletop Rearrangementcited by 6
  16. 2021Efficient and High-quality Prehensile Rearrangement in Cluttered and Confined Spacescited by 6
  17. 2021A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placementcited by 5
  18. 2023Minimizing Running Buffers for Tabletop Object Rearrangement: Complexity, Fast Algorithms, and Applicationscited by 4
  19. 2020Arranging Test Tubes in Racks Using Combined Task and Motion Planningcited by 3
  20. 2023Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesiscited by 3
  21. 2023Vanishing Bias Heuristic-guided Reinforcement Learning Algorithmcited by 2
  22. 2023In-Rack Test Tube Pose Estimation Using RGB-D Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtasks
  2. 2026Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning
  3. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
  4. 2026SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models
  5. 2024Synchronized Dual-arm Rearrangement via Cooperative mTSP
  6. 2024Learning Dual-arm Object Rearrangement for Cartesian Robots
  7. 2026Rainbow Deep Q-Learning with Kinematics-Aware Design for Cooperative Delta and 3-RRS Parallel Robot Insertion
  8. 2026LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts