mlsanityData to 7 Oct 2026

NeRP: Neural Rearrangement Planning for Unknown Objects

arXiv 2106.01352 ↗First posted 2 Jun 2021cs.RO cs.AI cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Robots will be expected to manipulate a wide variety of objects in complex and arbitrary ways as they become more widely used in human environments. As such, the rearrangement of objects has been noted to be an important benchmark for AI capabilities in recent years. We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world. We compare NeRP to several naive and model-based baselines, demonstrating that our approach is measurably better and can efficiently arrange unseen objects in fewer steps and with less planning time. Finally, we demonstrate it on several challenging rearrangement problems in the real world.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

30 of 38 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  8. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  9. 2020Integrated Task and Motion Planningcited by 281
  10. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 237
  11. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  12. 2020Rearrangement: A Challenge for Embodied AIcited by 153
18 more
  1. 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
  2. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  3. 2019Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Plannerscited by 62
  4. 2018Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstrationcited by 60
  5. 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 60
  6. 2017QMDP-Net: Deep Learning for Planning under Partial Observabilitycited by 51
  7. 2020Learning Physical Graph Representations from Visual Scenescited by 50
  8. 2020Object Rearrangement Using Learned Implicit Collision Functionscited by 43
  9. 2020Deep Affordance Foresight: Planning Through What Can Be Done in the Futurecited by 41
  10. 2019The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentationcited by 39
  11. 2018Learning to guide task and motion planning using score-space representationcited by 27
  12. 2020Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planningcited by 26
  13. 2019Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planningcited by 23
  14. 2020A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objectscited by 22
  15. 2020Neural Manipulation Planning on Constraint Manifoldscited by 15
  16. 2018Visual Robot Task Planningcited by 11
  17. 2020Constrained Motion Planning Networks Xcited by 9
  18. 2019Planning for target retrieval using a robotic manipulator in cluttered and occluded environmentscited by 4

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 8; 2023: 14; 2024: 4; 2025: 4.

33 papers.

Sort:

By area:

  1. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  2. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  3. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  4. 2022Motion Policy Networkscited by 53
  5. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  6. 2022StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objectscited by 49
  7. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  8. 2022Equivariant Transporter Networkcited by 44
  9. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  10. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  11. 2021Predicting Stable Configurations for Semantic Placement of Novel Objectscited by 23
  12. 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
21 more
  1. 2023DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observabilitycited by 21
  2. 2023Leveraging Symmetries in Pick and Placecited by 18
  3. 2023CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generationcited by 17
  4. 2022Transporters with Visual Foresight for Solving Unseen Rearrangement Taskscited by 9
  5. 2023Multi-Object Manipulation via Object-Centric Neural Scattering Functionscited by 9
  6. 2024"Set It Up!": Functional Object Arrangement with Compositional Generative Modelscited by 9
  7. 2023Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planningcited by 7
  8. 2022A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Searchcited by 6
  9. 2023Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Renderingcited by 5
  10. 2023Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Modelscited by 5
  11. 2024Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Priorcited by 5
  12. 2023Blocks2World: Controlling Realistic Scenes with Editable Primitivescited by 4
  13. 2025Cascaded Diffusion Models for Neural Motion Planningcited by 4
  14. 2023Spatial-Language Attention Policies for Efficient Robot Learningcited by 3
  15. 2025Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Cluttercited by 3
  16. 2022Robot Active Neural Sensing and Planning in Unknown Cluttered Environmentscited by 1
  17. 2023"Tidy Up the Table": Grounding Common-sense Objective for Tabletop Object Rearrangementcited by 1
  18. 2025Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangementcited by 1
  19. 2023MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered Environmentscited by 0
  20. 2024PACA: Perspective-Aware Cross-Attention Representation for Zero-Shot Scene Rearrangementcited by 0
  21. 2025Dynamic Buffers: Cost-Efficient Planning for Tabletop Rearrangement with Stackingcited by 0