mlsanityData to 7 Oct 2026

Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching

arXiv 1710.01330 ↗First posted 3 Oct 2017cs.RO cs.CVcited by 124

Not on the map. Why?

Abstract (arXiv)

This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a category-agnostic affordance prediction algorithm to select and execute among four different grasping primitive behaviors. It then recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional training data. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT-Princeton Team system that took 1st place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.edu

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 27; 2022: 17; 2023: 35; 2024: 16; 2025: 17; 2026: 11.

124 papers.

Sort:

By area:

  1. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  2. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  3. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  4. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  5. 2023Language-Driven Representation Learning for Roboticscited by 163
  6. 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
  7. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  8. 2021UMPNet: Universal Manipulation Policy Network for Articulated Objectscited by 64
  9. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 59
  10. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  11. 2024VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulationcited by 45
  12. 2022Equivariant Transporter Networkcited by 44
38 more
  1. 2021GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completioncited by 41
  2. 2022Differentiable Simulation of Soft Multi-body Systemscited by 39
  3. 2024Learning Manipulation by Predicting Interactioncited by 36
  4. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  5. 2023A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Cluttercited by 33
  6. 2021SuctionNet-1Billion: A Large-Scale Benchmark for Suction Graspingcited by 32
  7. 2021INVIGORATE: Interactive Visual Grounding and Grasping in Cluttercited by 32
  8. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
  9. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  10. 2022Affordance Learning from Play for Sample-Efficient Policy Learningcited by 27
  11. 2023Learning Foresightful Dense Visual Affordance for Deformable Object Manipulationcited by 25
  12. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Controlcited by 24
  13. 2023Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusionscited by 23
  14. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  15. 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
  16. 2022Interactron: Embodied Adaptive Object Detectioncited by 21
  17. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  18. 2022Visuotactile Affordances for Cloth Manipulation with Local Controlcited by 20
  19. 2023ReorientDiff: Diffusion Model based Reorientation for Object Manipulationcited by 19
  20. 2023Leveraging Symmetries in Pick and Placecited by 18
  21. 2023TRansPose: Large-Scale Multispectral Dataset for Transparent Objectcited by 15
  22. 2022Tac2Pose: Tactile Object Pose Estimation from the First Touchcited by 14
  23. 2024Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Cloudscited by 14
  24. 2022MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesiscited by 13
  25. 2023Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flightcited by 11
  26. 2025IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environmentscited by 11
  27. 2021Action Priors for Large Action Spaces in Roboticscited by 10
  28. 2022ReorientBot: Learning Object Reorientation for Specific-Posed Placementcited by 10
  29. 2023Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environmentcited by 10
  30. 2023Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspectivecited by 10
  31. 2021Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Modelscited by 9
  32. 2022Transporters with Visual Foresight for Solving Unseen Rearrangement Taskscited by 9
  33. 2023RGB-D Grasp Detection via Depth Guided Learning with Cross-modal Attentioncited by 9
  34. 2025GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulationcited by 9
  35. 2021GKNet: grasp keypoint network for grasp candidates detectioncited by 8
  36. 2021Multi-Task Learning with Sequence-Conditioned Transporter Networkscited by 8
  37. 2026AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understandingcited by 8
  38. 2021Contrastively Learning Visual Attention as Affordance Cues from Demonstrations for Robotic Graspingcited by 7

50 most cited of 124 All 124 (JSON)