Large Pre-Trained Models for Bimanual Manipulation in 3D
Not on the map. Why?
Abstract (arXiv)
We investigate the integration of attention maps from a pre-trained Vision Transformer into voxel representations to enhance bimanual robotic manipulation. Specifically, we extract attention maps from DINOv2, a self-supervised ViT model, and interpret them as pixel-level saliency scores over RGB images. These maps are lifted into a 3D voxel grid, resulting in voxel-level semantic cues that are incorporated into a behavior cloning policy. When integrated into a state-of-the-art voxel-based policy, our attention-guided featurization yields an average absolute improvement of 8.2% and a relative gain of 21.9% across all tasks in the RLBench bimanual benchmark.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 24 Sep 2025.
Builds on
34 of 49 references.
References · 34
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2023Vision Transformers Need Registerscited by 871
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
22 more
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
- 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
- 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
- 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 225
- 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 138
- 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
- 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
- 2024DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Modelscited by 44
- 2024Instant Policy: In-Context Imitation Learning via Graph Diffusioncited by 39
- 2024PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Taskscited by 37
- 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
- 2025Spatial-Temporal Graph Diffusion Policy with Kinematic Modeling for Bimanual Robotic Manipulationcited by 24
- 2025Locate 3D: Real-World Object Localization via Self-Supervised Learning in 3Dcited by 23
- 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
- 2025Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learningcited by 17
- 2025Gripper Keypose and Object Pointflow as Interfaces for Bimanual Robotic Manipulationcited by 16
- 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
- 2024TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approachcited by 5
- 2023Generalizable Imitation Learning Through Pre-Trained Representationscited by 3
- 2016Knowledge-Enabled Robotic Agents for Shelf Replenishment in Cluttered Retail Environmentscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.