mlsanityData to 7 Oct 2026

Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling

arXiv 2211.06407 ↗First posted 11 Nov 2022cs.RO cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Learning long-horizon tasks such as navigation has presented difficult challenges for successfully applying reinforcement learning to robotics. From another perspective, under known environments, sampling-based planning can robustly find collision-free paths in environments without learning. In this work, we propose Control Transformer that models return-conditioned sequences from low-level policies guided by a sampling-based Probabilistic Roadmap (PRM) planner. We demonstrate that our framework can solve long-horizon navigation tasks using only local information. We evaluate our approach on partially-observed maze navigation with MuJoCo robots, including Ant, Point, and Humanoid. We show that Control Transformer can successfully navigate through mazes and transfer to unknown environments. Additionally, we apply our method to a differential drive robot (Turtlebot3) and show zero-shot sim2real transfer under noisy observations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 53 references that are in the map. First posted on arXiv 11 Nov 2022.

Builds on

33 of 53 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  11. 2016The Option-Critic Architecturecited by 433
  12. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
21 more
  1. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  2. 2016Variational Intrinsic Controlcited by 262
  3. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  4. 2022Multi-Game Decision Transformerscited by 212
  5. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  6. 2022Online Decision Transformercited by 194
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  8. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  9. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  10. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  11. 2019Planning with Goal-Conditioned Policiescited by 144
  12. 2020Model-Based Offline Planningcited by 126
  13. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  14. 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
  15. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  16. 2019Reward-Conditioned Policiescited by 101
  17. 2017Learning Sampling Distributions for Robot Motion Planningcited by 79
  18. 2018Motion Planning Networkscited by 78
  19. 2022Can Wikipedia Help Offline Reinforcement Learning?cited by 76
  20. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 64
  21. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 1.

5 papers.

  1. 2023Large Language Models for Robotics: A Surveycited by 132
  2. 2024Potential Based Diffusion Motion Planningcited by 40
  3. 2023Merging Decision Transformers: Weight Averaging for Forming Multi-Task Policiescited by 15
  4. 2025Deep Active Inference with Diffusion Policy and Multiple Timescale World Model for Real-World Exploration and Navigationcited by 1
  5. 2026NaviFormer: A Deep Reinforcement Learning Transformer-like Model to Holistically Solve the Navigation Problemcited by 0