mlsanityData to 7 Oct 2026

ManipulaTHOR: A Framework for Visual Object Manipulation

arXiv 2104.11213 ↗First posted 22 Apr 2021cs.CV cs.AI cs.LGcited by 101

Not on the map. Why?

Abstract (arXiv)

The domain of Embodied AI has recently witnessed substantial progress, particularly in navigating agents within their environments. These early successes have laid the building blocks for the community to tackle tasks that require agents to actively interact with objects in their environment. Object manipulation is an established research domain within the robotics community and poses several challenges including manipulator motion, grasping and long-horizon planning, particularly when dealing with oft-overlooked practical setups involving visually rich and complex scenes, manipulation using mobile agents (as opposed to tabletop manipulation), and generalization to unseen environments and objects. We propose a framework for object manipulation built upon the physics-enabled, visually rich AI2-THOR framework and present a new challenge to the Embodied AI community known as ArmPointNav. This task extends the popular point navigation task to object manipulation and offers new challenges including 3D obstacle avoidance, manipulating objects in the presence of occlusion, and multi-object manipulation that necessitates long term planning. Popular learning paradigms that are successful on PointNav challenges show promise, but leave a large room for improvement.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 35 of its 42 references that are in the map. First posted on arXiv 22 Apr 2021.

Builds on

35 of 42 references.

References · 35

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  3. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  4. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  8. 2018On Evaluation of Embodied Navigation Agentscited by 526
  9. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  10. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  11. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  12. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
23 more
  1. 2018Semi-parametric Topological Memory for Navigationcited by 203
  2. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  3. 2016Learning to Navigate in Complex Environmentscited by 192
  4. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  5. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
  6. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 138
  7. 2018Visual Semantic Navigation using Scene Priorscited by 127
  8. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  9. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
  10. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
  11. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  12. 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
  13. 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
  14. 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 60
  15. 2020AllenAct: A Framework for Embodied AI Researchcited by 54
  16. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 52
  17. 2018Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervisioncited by 50
  18. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  19. 2017Visual Semantic Planning using Deep Successor Representationscited by 45
  20. 2020Learning 3D Dynamic Scene Representations for Robot Manipulationcited by 39
  21. 2017Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulationcited by 35
  22. 2016Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Searchcited by 25
  23. 2019Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 22; 2023: 26; 2024: 25; 2025: 15; 2026: 6.

101 papers.

Sort:

By area:

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  2. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  3. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  4. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  5. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  6. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  7. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  8. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
  9. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  10. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  11. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  12. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
38 more
  1. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  2. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  3. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  4. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  5. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  6. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  7. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  8. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  9. 20253D and 4D World Modeling: A Surveycited by 64
  10. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  11. 2024GRUtopia: Dream General Robots in a City at Scalecited by 54
  12. 2022Factory: Fast Contact for Robotic Assemblycited by 49
  13. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  14. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 47
  15. 2022Multi-skill Mobile Manipulation for Object Rearrangementcited by 46
  16. 2022Retrospectives on the Embodied AI Workshopcited by 46
  17. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  18. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  19. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  20. 2022Continuous Scene Representations for Embodied AIcited by 39
  21. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  22. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  23. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  24. 2023Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agentscited by 33
  25. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  26. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  27. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  28. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 26
  29. 2024UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AIcited by 25
  30. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 24
  31. 2025Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loopcited by 23
  32. 2024S2O: Static to Openable Enhancement for Articulated 3D Objectscited by 22
  33. 2024Online Continual Learning For Interactive Instruction Following Agentscited by 21
  34. 2025Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learningcited by 21
  35. 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AIcited by 19
  36. 2024MetaUrban: An Embodied AI Simulation Platform for Urban Micromobilitycited by 19
  37. 2022Ask4Help: Learning to Leverage an Expert for Embodied Taskscited by 18
  38. 2023CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generationcited by 17

50 most cited of 101 All 101 (JSON)