mlsanityData to 13 Oct 2026

GridToPix: Training Embodied Agents with Minimal Supervision

arXiv 2105.00931 ↗First posted 14 Apr 2021cs.CV cs.AI cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

While deep reinforcement learning (RL) promises freedom from hand-labeled data, great successes, especially for Embodied AI, require significant work to create supervision via carefully shaped rewards. Indeed, without shaped rewards, i.e., with only terminal rewards, present-day Embodied AI results degrade significantly across Embodied AI problems from single-agent Habitat-based PointGoal Navigation (SPL drops from 55 to 0) and two-agent AI2-THOR-based Furniture Moving (success drops from 58% to 1%) to three-agent Google Football-based 3 vs. 1 with Keeper (game score drops from 0.6 to 0.1). As training from shaped rewards doesn't scale to more realistic tasks, the community needs to improve the success of training with terminal rewards. For this we propose GridToPix: 1) train agents with terminal rewards in gridworlds that generically mirror Embodied AI environments, i.e., they are independent of the task; 2) distill the learned policy into agents that reside in complex visual worlds. Despite learning from only terminal rewards with identical models and RL algorithms, GridToPix significantly improves results across tasks: from PointGoal Navigation (SPL improves from 0 to 64) and Furniture Moving (success improves from 1% to 25%) to football gameplay (game score improves from 0.1 to 0.6). GridToPix even helps to improve the results of shaped reward training.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 61 of its 86 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

61 of 86 references.

References · 61

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,223
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,360
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,180
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,175
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,169
  8. 2017Hindsight Experience Replaycited by 1,104
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
  10. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 859
  11. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 786
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 775
38 more
  1. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 676
  2. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 588
  3. 2018On Evaluation of Embodied Navigation Agentscited by 531
  4. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 433
  6. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  7. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 403
  8. 2019Learning by Cheatingcited by 361
  9. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 353
  10. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  11. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  12. 2017Embodied Question Answeringcited by 335
  13. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 310
  14. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  15. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 259
  16. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 258
  17. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  18. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  19. 2019Vision-and-Dialog Navigationcited by 246
  20. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  21. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  22. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 186
  23. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  24. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 172
  25. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  26. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  27. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 104
  28. 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
  29. 2021Visual Room Rearrangementcited by 101
  30. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  31. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
  32. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  33. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  34. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 70
  35. 2020Semantic Visual Navigation by Watching YouTube Videoscited by 60
  36. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 57
  37. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  38. 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 54

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 6; 2024: 2; 2025: 1; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 222
  2. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 221
  3. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  4. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  5. 2022Retrospectives on the Embodied AI Workshopcited by 46
  6. 2023An Unbiased Look at Datasets for Visuo-Motor Pre-Trainingcited by 42
  7. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  8. 2022Last-Mile Embodied Visual Navigationcited by 25
  9. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 25
  10. 2022Learning Active Camera for Multi-Object Navigationcited by 20
  11. 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
  12. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
8 more
  1. 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14
  2. 2024Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Modelscited by 12
  3. 2023Exploitation-Guided Exploration for Semantic Embodied Navigationcited by 9
  4. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  5. 2022A General Purpose Supervisory Signal for Embodied Agentscited by 3
  6. 2025Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learningcited by 3
  7. 2026Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actionscited by 1
  8. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0