mlsanityData to 7 Oct 2026

Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians

arXiv 2102.13073 ↗First posted 25 Feb 2021cs.RO cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Robotic navigation in environments shared with other robots or humans remains challenging because the intentions of the surrounding agents are not directly observable and the environment conditions are continuously changing. Local trajectory optimization methods, such as model predictive control (MPC), can deal with those changes but require global guidance, which is not trivial to obtain in crowded scenarios. This paper proposes to learn, via deep Reinforcement Learning (RL), an interaction-aware policy that provides long-term guidance to the local planner. In particular, in simulations with cooperative and non-cooperative agents, we train a deep network to recommend a subgoal for the MPC planner. The recommended subgoal is expected to help the robot in making progress towards its goal and accounts for the expected interaction with other agents. Based on the recommended subgoal, the MPC planner then optimizes the inputs for the robot satisfying its kinodynamic and collision avoidance constraints. Our approach is shown to substantially improve the navigation performance in terms of number of collisions as compared to prior MPC frameworks, and in terms of both travel time and number of collisions compared to deep RL methods in cooperative, competitive and mixed multiagent scenarios.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

20 of 44 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  4. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  5. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 162
  6. 2017Socially Aware Motion Planning with Deep Reinforcement Learningcited by 128
  7. 2016Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learningcited by 104
  8. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 104
  9. 2019Exploring Model-based Planning with Policy Networkscited by 90
  10. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  11. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 50
  12. 2019Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learningcited by 44
8 more
  1. 2020Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environmentscited by 19
  2. 2019Practical Reinforcement Learning of Stabilizing Economic MPCcited by 15
  3. 2020Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robotcited by 15
  4. 2019Deep Value Model Predictive Controlcited by 14
  5. 2017Modeling Cooperative Navigation in Dense Human Crowdscited by 9
  6. 2018Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarioscited by 4
  7. 2019Model-based Lookahead Reinforcement Learningcited by 4
  8. 2019Combining Benefits from Trajectory Optimization and Deep Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 4; 2025: 3; 2026: 4.

20 papers.

Sort:

By area:

  1. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 19
  2. 2025LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generationcited by 13
  3. 2024SoNIC: Safe Social Navigation with Adaptive Conformal Inference and Constrained Reinforcement Learningcited by 8
  4. 2024Socially Integrated Navigation: A Social Acting Robot with Deep Reinforcement Learningcited by 5
  5. 2022Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Controlcited by 4
  6. 2023Reinforcement Learning in Robotic Motion Planning by Combined Experience-based Planning and Self-Imitation Learningcited by 4
  7. 2023Learning-Aided Warmstart of Model Predictive Control in Uncertain Fast-Changing Trafficcited by 3
  8. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  9. 2021Reinforcement Learning for Robot Navigation with Adaptive Forward Simulation Time (AFST) in a Semi-Markov Modelcited by 2
  10. 2023Holistic Deep-Reinforcement-Learning-based Training of Autonomous Navigation Systemscited by 2
  11. 2024Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learningcited by 2
  12. 2022Multi-subgoal Robot Navigation in Crowds with History Information and Interactionscited by 1
8 more
  1. 2023Virtual Guidance as a Mid-level Representation for Navigation with Augmented Realitycited by 1
  2. 2023Evolutionary Curriculum Training for DRL-Based Navigation Systemscited by 1
  3. 2023GP-guided MPPI for Efficient Navigation in Complex Unknown Cluttered Environmentscited by 0
  4. 2024Improving the Generalization of Unseen Crowd Behaviors for Reinforcement Learning based Local Motion Plannerscited by 0
  5. 2026PIER-Flow: Physics-Informed Efficient Rectified Flow for Real-Time Mobile Robot Navigationcited by 0
  6. 2026Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policiescited by 0
  7. 2026Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigmscited by 0
  8. 2026SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Controlcited by 0