mlsanityData to 13 Oct 2026

Reinforcement Learning for Robot Navigation with Adaptive Forward Simulation Time (AFST) in a Semi-Markov Model

arXiv 2108.06161 ↗First posted 13 Aug 2021cs.RO cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) algorithms have proven effective in robot navigation, especially in unknown environments, by directly mapping perception inputs into robot control commands. However, most existing methods ignore the local minimum problem in navigation and thereby cannot handle complex unknown environments. In this paper, we propose the first DRL-based navigation method modeled by a semi-Markov decision process (SMDP) with continuous action space, named Adaptive Forward Simulation Time (AFST), to overcome this problem. Specifically, we reduce the dimensions of the action space and improve the distributed proximal policy optimization (DPPO) algorithm for the specified SMDP problem by modifying its GAE to better estimate the policy gradient in SMDPs. Experiments in various unknown environments demonstrate the effectiveness of AFST.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 13 of its 35 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

13 of 35 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,940
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  4. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  5. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  6. 2016Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learningcited by 104
  7. 2018Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learningcited by 101
  8. 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 91
  9. 2017Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learningcited by 38
  10. 2018Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrationscited by 35
  11. 2021Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrianscited by 20
  12. 2016Deep Reinforcement Learning With Macro-Actionscited by 13
1 more
  1. 2021DRQN-based 3D Obstacle Avoidance with a Limited Field of Viewcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systemscited by 0
  2. 2026Temporal Self-Imitation Learningcited by 0