mlsanityData to 7 Oct 2026

PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators

arXiv 2406.20083 ↗First posted 28 Jun 2024cs.RO cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 72 of its 86 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

72 of 86 references.

References · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Segment Anythingcited by 8,158
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  11. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  12. 2020Rethinking Attention with Performerscited by 1,584
38 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2022A Generalist Agentcited by 802
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  8. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  9. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  10. 2022Efficiently Scaling Transformer Inferencecited by 529
  11. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  12. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  13. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  14. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  15. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  16. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  17. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  18. 2020Learning to Explore using Active Neural SLAMcited by 308
  19. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  20. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  21. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  22. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  23. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
  24. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  25. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  26. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  27. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  28. 2022Online Decision Transformercited by 194
  29. 2020Neural Topological SLAM for Visual Navigationcited by 193
  30. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  31. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  32. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  33. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  34. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  35. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  36. 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
  37. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  38. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 28; 2026: 20.

55 papers.

Sort:

By area:

  1. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  2. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  3. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
  4. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  5. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 63
  6. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  7. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  8. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  9. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  10. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  11. 2024DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulationcited by 32
  12. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
38 more
  1. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  2. 2025Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learningcited by 21
  3. 2025TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Trackingcited by 16
  4. 2025SIMS-V: Simulated Instruction-Tuning for Spatial Video Understandingcited by 16
  5. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  6. 2025Do Visual Imaginations Improve Vision-and-Language Navigation Agents?cited by 14
  7. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  8. 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 10
  9. 2024FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigationcited by 9
  10. 2025ForesightNav: Learning Scene Imagination for Efficient Explorationcited by 9
  11. 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigationcited by 7
  12. 2025View Invariant Learning for Vision-Language Navigation in Continuous Environmentscited by 6
  13. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  14. 2026LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigationcited by 5
  15. 2026CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Modelscited by 5
  16. 2026ABot-N1: Toward a General Visual Language Navigation Foundation Modelcited by 5
  17. 2025Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approachcited by 4
  18. 2025Let Humanoids Hike! Integrative Skill Development on Complex Trailscited by 4
  19. 2025Constrained Decoding for Safe Robot Navigation Foundation Modelscited by 3
  20. 2025MetricNet: Recovering Metric Scale in Generative Navigation Policiescited by 3
  21. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  22. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontierscited by 3
  23. 2025Humans Coexist, So Must Embodied Artificial Agentscited by 2
  24. 2025RANa: Retrieval-Augmented Navigationcited by 2
  25. 2025What does really matter in image goal navigation?cited by 2
  26. 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Modelscited by 2
  27. 2025Kinaema: a recurrent sequence model for memory and pose in motioncited by 2
  28. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigationcited by 2
  29. 2026Embodiment-Aware Generalist Specialist Distillation for Unified Humanoid Whole-Body Controlcited by 2
  30. 2026NavOL: Navigation Policy with Online Imitation Learningcited by 2
  31. 2025Synthetic vs. Real Training Data for Visual Navigationcited by 1
  32. 2026USS: Unifying Spatial-Semantic Prompting for End to End Embodied Visual Trackingcited by 1
  33. 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1
  34. 2025Data-Efficient Multitask DAggercited by 0
  35. 2025RANGER: A Monocular Zero-Shot Semantic Navigation Framework through Visual Contextual Adaptationcited by 0
  36. 2025DRL-TH: Jointly Utilizing Temporal Graph Attention and Hierarchical Fusion for UGV Navigation in Crowded Environmentscited by 0
  37. 2026Specification-Aware Distribution Shaping for Robotics Foundation Modelscited by 0
  38. 2026StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigationcited by 0

50 most cited of 55 All 55 (JSON)