# Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviors

arXiv: https://arxiv.org/abs/2103.03793 · HTML: https://mlsanity.com/paper/2103.03793/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [IPO: Interior-point Policy Optimization under Constraints](https://mlsanity.com/paper/1910.09615/) (2019)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization](https://mlsanity.com/paper/2002.11635/) (2020)
- [SAMBA: Safe Model-Based & Active Reinforcement Learning](https://mlsanity.com/paper/2006.09436/) (2020)
- [Learning Robotic Assembly from CAD](https://mlsanity.com/paper/1803.07635/) (2018)
- [Safe Certificate-Based Maneuvers for Teams of Quadrotors Using Differential Flatness](https://mlsanity.com/paper/1702.01075/) (2017)
- [Learning Robot Trajectories subject to Kinematic Joint Constraints](https://mlsanity.com/paper/2011.00563/) (2020)
- [TrueAEdapt: Learning Smooth Online Trajectory Adaptation with Bounded Jerk, Acceleration and Velocity in Joint Space](https://mlsanity.com/paper/2006.00375/) (2020)

## Built on by (most cited first, 4 of 4)

- [Combining Deep Reinforcement Learning with a Jerk-Bounded Trajectory Generator for Kinematically Constrained Motion Planning](https://mlsanity.com/paper/2410.20907/) (2024)
- [Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles](https://mlsanity.com/paper/2411.05784/) (2024)
- [Learning Time-optimized Path Tracking with or without Sensory Feedback](https://mlsanity.com/paper/2203.01968/) (2022)
- [CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrains](https://mlsanity.com/paper/2602.03511/) (2026)

## Abstract (arXiv)

This paper presents an approach for learning online generation of collision-free and torque-limited robot trajectories. In order to generate future motions, a neural network is periodically invoked. Based on the current kinematic state of the robot and the network output, a trajectory for the current time interval can be calculated. The main idea of our paper is to execute the computed motion only if a collision-free and torque-limited way to continue the trajectory is known. In practice, the motion computed for the current time interval is extended by a braking trajectory and simulated using a physics engine. If the simulated trajectory complies with all safety constraints, the computed motion is carried out. Otherwise, the braking trajectory calculated in the previous time interval serves as an alternative safe behavior. Given a task-specific reward function, the neural network is trained using reinforcement learning. The design of the action space used for reinforcement learning ensures that all computed trajectories comply with kinematic joint limits. For our evaluation, simulated humanoid robots and industrial robots are trained to reach as many randomly placed target points as possible. We show that our method reliably prevents collisions with static obstacles and collisions between the robot arms, while generating motions that respect both torque limits and kinematic joint limits. Experiments with a real robot demonstrate that safe trajectories can be generated in real-time.