mlsanityData to 14 Oct 2026

RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation

arXiv 2407.04689 ↗First posted 5 Jul 2024cs.RO cs.CVcited by 58

Not on the map. Why?

Abstract (arXiv)

This work proposes a retrieve-and-transfer framework for zero-shot robotic manipulation, dubbed RAM, featuring generalizability across various objects, environments, and embodiments. Unlike existing approaches that learn manipulation from expensive in-domain demonstrations, RAM capitalizes on a retrieval-based affordance transfer paradigm to acquire versatile manipulation capabilities from abundant out-of-domain data. First, RAM extracts unified affordance at scale from diverse sources of demonstrations including robotic data, human-object interaction (HOI) data, and custom data to construct a comprehensive affordance memory. Then given a language instruction, RAM hierarchically retrieves the most similar demonstration from the affordance memory and transfers such out-of-domain 2D affordance to in-domain 3D executable affordance in a zero-shot and embodiment-agnostic manner. Extensive simulation and real-world evaluations demonstrate that our RAM consistently outperforms existing works in diverse daily tasks. Additionally, RAM shows significant potential for downstream applications such as automatic and efficient data collection, one-shot visual imitation, and LLM/VLM-integrated long-horizon manipulation. For more details, please check our website at https://yxkryptonite.github.io/RAM/.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 63 of its 73 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

63 of 73 references.

References · 63

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  9. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  12. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
38 more
  1. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  2. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  4. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  5. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  6. 2023Emergent Correspondence from Image Diffusioncited by 446
  7. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  8. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  9. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  10. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  11. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  12. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  13. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  14. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
  15. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  16. 2021ASFormer: Transformer for Action Segmentationcited by 168
  17. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
  18. 2022Human-to-Robot Imitation in the Wildcited by 161
  19. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  20. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  21. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
  22. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 134
  23. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  24. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 124
  25. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 111
  26. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  27. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
  28. 2014Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Studycited by 89
  29. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  30. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  31. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  32. 2022Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applicationscited by 78
  33. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 71
  34. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 69
  35. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 67
  36. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 58
  37. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 57
  38. 2022End-to-End Affordance Learning for Robotic Manipulationcited by 57

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 24; 2026: 22.

58 papers.

Sort:

By area:

  1. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 130
  2. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 54
  3. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  4. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 45
  5. 2024STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learningcited by 44
  6. 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordancecited by 37
  7. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  8. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 32
  9. 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
  10. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 29
  11. 2024GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Groundingcited by 28
  12. 2025ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulationcited by 24
38 more
  1. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 21
  2. 2025AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Cluttercited by 20
  3. 2024GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Graspingcited by 15
  4. 2025GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulationcited by 14
  5. 2026Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulationcited by 14
  6. 2024One-Shot Imitation under Mismatched Executioncited by 12
  7. 2026AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understandingcited by 12
  8. 2025METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Modelcited by 10
  9. 2025GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulationcited by 9
  10. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blendingcited by 9
  11. 2024One-Shot Manipulation Strategy Learning by Making Contact Analogiescited by 8
  12. 2025Growing with Your Embodied Agent: A Human-in-the-Loop Lifelong Code Generation Framework for Long-Horizon Manipulation Skillscited by 7
  13. 2025Robot Learning from Any Imagescited by 7
  14. 2024EqvAfford: SE(3) Equivariance for Point-Level Affordance Learningcited by 6
  15. 2025RT-Cache: Training-Free Retrieval for Real-Time Manipulationcited by 6
  16. 2024GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulationcited by 5
  17. 2025IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Modelscited by 5
  18. 2025COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learningcited by 5
  19. 2026GeneralVLA: Generalizable Vision-Language-Action Models with Knowledge-Guided Trajectory Planningcited by 5
  20. 2026Robot Learning from Human Videos: A Surveycited by 5
  21. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Modelscited by 4
  22. 2025RealDrive: Retrieval-Augmented Driving with Diffusion Modelscited by 3
  23. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challengescited by 3
  24. 2025O3Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulationcited by 3
  25. 2025Dejavu: Towards Experience Feedback Learning for Embodied Intelligencecited by 3
  26. 2026SAIL: Test-Time Scaling for In-Context Imitation Learning with VLMcited by 3
  27. 2025Weakly-Supervised Learning of Dense Functional Correspondencescited by 2
  28. 2025Unlocking 3D Affordance Segmentation with 2D Semantic Knowledgecited by 2
  29. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 2
  30. 2026Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warpingcited by 2
  31. 2026PAWS: Perception of Articulation in the Wild at Scale from Egocentric Videoscited by 2
  32. 2026AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Afford Correspondencecited by 2
  33. 2026VLA-Pro: Cross-Task Procedural Memory Transfer for Vision-Language-Action Modelscited by 2
  34. 2024QuadWBG: Generalizable Quadrupedal Whole-Body Graspingcited by 1
  35. 2025Manipulate-to-Navigate: Reinforcement Learning with Visual Affordances and Manipulability Priorscited by 1
  36. 2026ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstrationcited by 1
  37. 2026Retrieve in Time, Correct in Frequencycited by 1
  38. 2026VLAff: Vision-Language-Affordance Model for Unified Actionable Affordancescited by 1

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Affordance RAG: Hierarchical Multimodal Retrieval with Affordance-Aware Embodied Memory for Mobile Manipulation
  2. 2026AffordTrajDP: Dynamic Affordance-Guided Visuomotor Policy Learning for Robotic Manipulation
  3. 2026AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation
  4. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
  5. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation
  6. 2025RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping
  7. 2026AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
  8. 2026Affordance Transfer Across Object Instances via Semantically Anchored Functional Map