LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation
Not on the map. Why?
Abstract (arXiv)
Human-like generalization in open-world remains a fundamental challenge for robotic manipulation. Existing learning-based methods, including reinforcement learning, imitation learning, and vision-language-action-models (VLAs), often struggle with novel tasks and unseen environments. Another promising direction is to explore generalizable representations that capture fine-grained spatial and geometric relations for open-world manipulation. While large-language-model (LLMs) and vision-language-model (VLMs) provide strong semantic reasoning based on language or annotated 2D representations, their limited 3D awareness restricts their applicability to fine-grained manipulation. To address this, we propose LAMP, which lifts image-editing as 3D priors to extract inter-object 3D transformations as continuous, geometry-aware representations. Our key insight is that image-editing inherently encodes rich 2D spatial cues, and lifting these implicit cues into 3D transformations provides fine-grained and accurate guidance for open-world manipulation. Extensive experiments demonstrate that codename delivers precise 3D transformations and achieves strong zero-shot generalization in open-world manipulation. Project page: https://zju3dv.github.io/LAMP/.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 85 of its 95 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
85 of 95 references.
References · 85
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Qwen Technical Reportcited by 3,273
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
38 more
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
- 2025DINOv3cited by 1,297
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2025Qwen-Image Technical Reportcited by 973
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
- 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025Seed1.5-VL Technical Reportcited by 367
- 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2019Deep Closest Point: Learning Representations for Point Cloud Registrationcited by 318
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
- 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
- 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
- 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
- 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
- 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 205
- 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
- 2023Putting the Object Back into Video Object Segmentationcited by 182
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
- 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
- 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
- 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
- 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
- 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
- 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
50 most cited of 85 All 85 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 20253D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks
- 2026BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
- 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
- 2026Dexterity-BEV: Aligning 3D World and Actions for Generalizable Robot Policies Learning
- 2026Robotic Manipulation is Vision-to-Geometry Mapping: Vision-Geometry Backbones over Language and Video Models
- 2024Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation
- 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation