# CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL

arXiv: https://arxiv.org/abs/2103.13267 · HTML: https://mlsanity.com/paper/2103.13267/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Improving Sample Efficiency in Model-Free Reinforcement Learning from Images](https://mlsanity.com/paper/1910.01741/) (2019)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Value Iteration Networks](https://mlsanity.com/paper/1602.02867/) (2016)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Hierarchical Imitation and Reinforcement Learning](https://mlsanity.com/paper/1803.00590/) (2018)
- [Learning Sampling Distributions for Robot Motion Planning](https://mlsanity.com/paper/1709.05448/) (2017)
- [Motion Planning Networks](https://mlsanity.com/paper/1806.05767/) (2018)
- [Robot Motion Planning in Learned Latent Spaces](https://mlsanity.com/paper/1807.10366/) (2018)
- [Data-driven Planning via Imitation Learning](https://mlsanity.com/paper/1711.06391/) (2017)
- [Universal Planning Networks](https://mlsanity.com/paper/1804.00645/) (2018)
- [Harnessing Reinforcement Learning for Neural Motion Planning](https://mlsanity.com/paper/1906.00214/) (2019)
- [Learning a Decentralized Multi-arm Motion Planner](https://mlsanity.com/paper/2011.02608/) (2020)
- [Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras](https://mlsanity.com/paper/2002.09107/) (2020)
- [Learning Vision-based Reactive Policies for Obstacle Avoidance](https://mlsanity.com/paper/2010.16298/) (2020)
- [Deep sequential models for sampling-based planning](https://mlsanity.com/paper/1810.00804/) (2018)
- [A Critical Investigation of Deep Reinforcement Learning for Navigation](https://mlsanity.com/paper/1802.02274/) (2018)
- [Addressing Sample Complexity in Visual Tasks Using HER and Hallucinatory GANs](https://mlsanity.com/paper/1901.11529/) (2019)
- [Asymptotically Optimal Sampling-based Planners](https://mlsanity.com/paper/1911.04044/) (2019)

## Built on by (most cited first, 1 of 1)

- [CalliRewrite: Recovering Handwriting Behaviors from Calligraphy Images without Supervision](https://mlsanity.com/paper/2405.15776/) (2024)

## Abstract (arXiv)

We propose a vision-based reinforcement learning (RL) approach for closed-loop trajectory generation in an arm reaching problem. Arm trajectory generation is a fundamental robotics problem which entails finding collision-free paths to move the robot's body (e.g. arm) in order to satisfy a goal (e.g. place end-effector at a point). While classical methods typically require the model of the environment to solve a planning, search or optimization problem, learning-based approaches hold the promise of directly mapping from observations to robot actions. However, learning a collision-avoidance policy using RL remains a challenge for various reasons, including, but not limited to, partial observability, poor exploration, low sample efficiency, and learning instabilities. To address these challenges, we present a residual-RL method that leverages a greedy goal-reaching RL policy as the base to improve exploration, and the base policy is augmented with residual state-action values and residual actions learned from images to avoid obstacles. Further more, we introduce novel learning objectives and techniques to improve 3D understanding from multiple image views and sample efficiency of our algorithm. Compared to RL baselines, our method achieves superior performance in terms of success rate.