mlsanityData to 7 Oct 2026

Rearrangement: A Challenge for Embodied AI

arXiv 2011.01975 ↗First posted 3 Nov 2020cs.AI cs.CV cs.LGcited by 153

Not on the map. Why?

Abstract (arXiv)

We describe a framework for research and evaluation in Embodied AI. Our proposal is based on a canonical task: Rearrangement. A standard task can focus the development of new techniques and serve as a source of trained models that can be transferred to other settings. In the rearrangement task, the goal is to bring a given physical environment into a specified state. The goal state can be specified by object poses, by images, by a description in language, or by letting the agent experience the environment in the goal state. We characterize rearrangement scenarios along different axes and describe metrics for benchmarking rearrangement performance. To facilitate research and exploration, we present experimental testbeds of rearrangement scenarios in four different simulation environments. We anticipate that other datasets will be released and new simulation platforms will be built to support training of rearrangement agents and their deployment on physical systems.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 37; 2023: 38; 2024: 33; 2025: 13; 2026: 6.

153 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  3. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  4. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  6. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  7. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  8. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  9. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  10. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  11. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  12. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
38 more
  1. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  2. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  3. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  4. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 105
  5. 2021Visual Room Rearrangementcited by 101
  6. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  7. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  8. 2023LEGO-Net: Learning Regular Rearrangements of Objects in Roomscited by 75
  9. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  10. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  11. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  12. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
  13. 2022TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphscited by 59
  14. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 57
  15. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  16. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  17. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  18. 2022OPD: Single-view 3D Openable Part Detectioncited by 52
  19. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  20. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  21. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  22. 2022Multi-skill Mobile Manipulation for Object Rearrangementcited by 46
  23. 2022Retrospectives on the Embodied AI Workshopcited by 46
  24. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  25. 2022Continuous Scene Representations for Embodied AIcited by 39
  26. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  27. 2022Understanding 3D Object Articulation in Internet Videoscited by 36
  28. 2021NeRP: Neural Rearrangement Planning for Unknown Objectscited by 33
  29. 2022Is Mapping Necessary for Realistic PointGoal Navigation?cited by 32
  30. 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
  31. 2023Emergence of Maps in the Memories of Blind Navigation Agentscited by 31
  32. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  33. 2022TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priorscited by 30
  34. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  35. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  36. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  37. 2021My House, My Rules: Learning Tidying Preferences with Graph Neural Networkscited by 27
  38. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 26

50 most cited of 153 All 153 (JSON)