# Learning Manipulation Tasks in Dynamic and Shared 3D Spaces

arXiv: https://arxiv.org/abs/2404.17673 · HTML: https://mlsanity.com/paper/2404.17673/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [SAM3D: Segment Anything in 3D Scenes](https://mlsanity.com/paper/2306.03908/) (2023)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment](https://mlsanity.com/paper/2302.10717/) (2023)
- [Constrained Probabilistic Movement Primitives for Robot Trajectory Adaptation](https://mlsanity.com/paper/2101.12561/) (2021)
- [Non-Prehensile Manipulation in Clutter with Human-In-The-Loop](https://mlsanity.com/paper/1904.03748/) (2019)
- [Coordination of two robotic manipulators for object retrieval in clutter](https://mlsanity.com/paper/2109.15220/) (2021)
- [Environment-aware Interactive Movement Primitives for Object Reaching in Clutter](https://mlsanity.com/paper/2210.16194/) (2022)

## Abstract (arXiv)

Automating the segregation process is a need for every sector experiencing a high volume of materials handling, repetitive and exhaustive operations, in addition to risky exposures. Learning automated pick-and-place operations can be efficiently done by introducing collaborative autonomous systems (e.g. manipulators) in the workplace and among human operators. In this paper, we propose a deep reinforcement learning strategy to learn the place task of multi-categorical items from a shared workspace between dual-manipulators and to multi-goal destinations, assuming the pick has been already completed. The learning strategy leverages first a stochastic actor-critic framework to train an agent's policy network, and second, a dynamic 3D Gym environment where both static and dynamic obstacles (e.g. human factors and robot mate) constitute the state space of a Markov decision process. Learning is conducted in a Gazebo simulator and experiments show an increase in cumulative reward function for the agent further away from human factors. Future investigations will be conducted to enhance the task performance for both agents simultaneously.