mlsanityData to 7 Oct 2026

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

arXiv 2202.00732 ↗First posted 1 Feb 2022cs.RO cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Accurate object rearrangement from vision is a crucial problem for a wide variety of real-world robotics applications in unstructured environments. We propose IFOR, Iterative Flow Minimization for Robotic Object Rearrangement, an end-to-end method for the challenging problem of object rearrangement for unknown objects given an RGBD image of the original and final scenes. First, we learn an optical flow model based on RAFT to estimate the relative transformation of the objects purely from synthetic data. This flow is then used in an iterative minimization algorithm to achieve accurate positioning of previously unseen objects. Crucially, we show that our method applies to cluttered scenes, and in the real world, while training only on synthetic data. Videos are available at https://imankgoyal.github.io/ifor.html.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 66 of its 76 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

66 of 76 references.

References · 66

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2017Mask R-CNNcited by 7,259
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  9. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  10. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  12. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
38 more
  1. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  2. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  3. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  4. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  5. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  6. 2017Embodied Question Answeringcited by 335
  7. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 324
  8. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  9. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  10. 2020Integrated Task and Motion Planningcited by 281
  11. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  12. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  13. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 237
  14. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  15. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  16. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  17. 2017Learning to Find Good Correspondencescited by 176
  18. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  19. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  20. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
  21. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  22. 2020Patch2Pix: Epipolar-Guided Pixel-Level Correspondencescited by 128
  23. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 121
  24. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 117
  25. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 108
  26. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  27. 2021Visual Room Rearrangementcited by 101
  28. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  29. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  30. 2020Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networkscited by 77
  31. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  32. 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
  33. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  34. 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 60
  35. 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
  36. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  37. 2019Deep Rigid Instance Scene Flowcited by 54
  38. 2020UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learningcited by 49

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 14; 2024: 6; 2025: 12; 2026: 5.

42 papers.

Sort:

By area:

  1. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  2. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  3. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  4. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  5. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  6. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 105
  7. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  8. 2022StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objectscited by 49
  9. 2024OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Roboticscited by 33
  10. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  11. 2025Pre-training Auto-regressive Robotic Models with 4D Representationscited by 27
  12. 2023SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphscited by 24
30 more
  1. 2025Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flowcited by 21
  2. 2022Mean Shift Mask Transformer for Unseen Object Instance Segmentationcited by 19
  3. 2023CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generationcited by 17
  4. 2025TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videoscited by 13
  5. 2023Self-Supervised Motion Magnification by Backpropagating Through Optical Flowcited by 10
  6. 2024Never-Ending Behavior-Cloning Agent for Robotic Manipulationcited by 10
  7. 2025GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulationcited by 9
  8. 2025Latent Policy Steering with Embodiment-Agnostic Pretrained World Modelscited by 8
  9. 2025H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videoscited by 8
  10. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 8
  11. 2023Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planningcited by 7
  12. 2022A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Searchcited by 6
  13. 2024Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Priorcited by 5
  14. 2023STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robotscited by 4
  15. 2023SceneScore: Learning a Cost Function for Object Arrangementcited by 4
  16. 2024Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Datacited by 4
  17. 2025Cascaded Diffusion Models for Neural Motion Planningcited by 4
  18. 2023Spatial-Language Attention Policies for Efficient Robot Learningcited by 3
  19. 2023Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planningcited by 3
  20. 2025Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Cluttercited by 3
  21. 2023High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinementcited by 1
  22. 2025Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangementcited by 1
  23. 2026GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoningcited by 1
  24. 2026HyReach: Vision-Guided Hybrid Manipulator Reaching in Unseen Cluttered Environmentscited by 1
  25. 2023Fully Automated Task Management for Generation, Execution, and Evaluation: A Framework for Fetch-and-Carry Tasks with Natural Language Instructions in Continuous Spacecited by 0
  26. 2025Temporal Prompting Matters: Rethinking Referring Video Object Segmentationcited by 0
  27. 2025ArtiBench and ArtiBrain: Benchmarking Generalizable Vision-Language Articulated Object Manipulationcited by 0
  28. 2025Explaining Why Things Go Where They Go: Interpretable Constructs of Human Organizational Preferencescited by 0
  29. 2026GraspLDP: Towards Generalizable Grasping Policy via Latent Diffusioncited by 0
  30. 2026A Cross-view Fusion Framework for Robust 6-DoF Grasp Pose Estimationcited by 0