# Trajectory Planning with Deep Reinforcement Learning in High-Level Action Spaces

arXiv: https://arxiv.org/abs/2110.00044 · HTML: https://mlsanity.com/paper/2110.00044/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2021.

## Built on by (most cited first, 5 of 5)

- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions](https://mlsanity.com/paper/2402.04168/) (2024)
- [Rethinking Lanes and Points in Complex Scenarios for Monocular 3D Lane Detection](https://mlsanity.com/paper/2503.06237/) (2025)
- [Scalable Volt-VAR Optimization using RLlib-IMPALA Framework: A Reinforcement Learning Approach](https://mlsanity.com/paper/2402.15932/) (2024)
- [Breaking Down Monocular Ambiguity: Exploiting Temporal Evolution for 3D Lane Detection](https://mlsanity.com/paper/2504.20525/) (2025)

## Abstract (arXiv)

This paper presents a technique for trajectory planning based on continuously parameterized high-level actions (motion primitives) of variable duration. This technique leverages deep reinforcement learning (Deep RL) to formulate a policy which is suitable for real-time implementation. There is no separation of motion primitive generation and trajectory planning: each individual short-horizon motion is formed during the Deep RL training to achieve the full-horizon objective. Effectiveness of the technique is demonstrated numerically on a well-studied trajectory generation problem and a planning problem on a known obstacle-rich map. This paper also develops a new loss function term for policy-gradient-based Deep RL, which is analogous to an anti-windup mechanism in feedback control. We demonstrate the inclusion of this new term in the underlying optimization increases the average policy return in our numerical example.