Self-supervised 6D Object Pose Estimation for Robot Manipulation
Not on the map. Why?
Abstract (arXiv)
To teach robots skills, it is crucial to obtain data with supervision. Since annotating real world data is time-consuming and expensive, enabling robots to learn in a self-supervised way is important. In this work, we introduce a robot system for self-supervised 6D object pose estimation. Starting from modules trained in simulation, our system is able to label real world images with accurate 6D object poses for self-supervised learning. In addition, the robot interacts with objects in the environment to change the object configuration by grasping or pushing objects. In this way, our system is able to continuously collect data and improve its pose estimation modules. We show that the self-supervised learning improves object segmentation and 6D pose estimation performance, and consequently enables the system to grasp objects more reliably. A video showing the experiments can be found at https://youtu.be/W1Y0Mmh1Gd8.
Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 17; 2022: 24; 2023: 20; 2024: 18; 2025: 15; 2026: 6.
100 papers.
Sort:
By area:
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
- 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
- 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
- 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
- 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 54
- 2021SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimationcited by 53
- 2023GenPose: Generative Category-level Object Pose Estimation via Diffusion Modelscited by 47
- 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
- 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
38 more
- 2022Equivariant Transporter Networkcited by 44
- 2022RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimationcited by 43
- 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
- 2022SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimationcited by 41
- 2023IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformationcited by 40
- 2023Learning Object State Changes in Videos: An Open-World Perspectivecited by 36
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
- 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordancecited by 35
- 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
- 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
- 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
- 2021Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overviewcited by 31
- 2022TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimationcited by 23
- 2022Visuomotor Control in Multi-Object Scenes Using Object-Aware Representationscited by 21
- 2021DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstrationcited by 20
- 2022Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Pickingcited by 20
- 2021PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learningcited by 19
- 2022Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructionscited by 19
- 2023ReorientDiff: Diffusion Model based Reorientation for Object Manipulationcited by 19
- 2023Leveraging Symmetries in Pick and Placecited by 18
- 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulationcited by 18
- 2021Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transformscited by 17
- 2023Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimationcited by 16
- 2022Transformers are Adaptable Task Plannerscited by 15
- 2021Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimationcited by 14
- 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speedcited by 14
- 2024GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splattingcited by 14
- 2024LaPose: Laplacian Mixture Shape Modeling for RGB-Based Category-Level Object Pose Estimationcited by 13
- 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
- 2023Self-Supervised Unseen Object Instance Segmentation via Long-Term Robot Interactioncited by 10
- 2023MFOS: Model-Free & One-Shot Object Pose Estimationcited by 10
- 20246DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splattingcited by 10
- 2021Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Cloudscited by 9
- 2021IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenescited by 9
- 2022Mobile Robot Manipulation using Pure Object Detectioncited by 8
- 2023Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priorscited by 8
- 2021Differentiable Rendering with Perturbed Optimizerscited by 7
- 2021Depth from Camera Motion and Object Detectioncited by 6
50 most cited of 100 All 100 (JSON)
- 2026G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation20 Sep 2026
- 2026Learning to Throw Objects Safely in Multi-Obstacle Environments7 Jul 2026
- 2026Efficiently Linking Real Scenes with Synthetic Data Generation for AI-based Cognitive Robotics and Computer Vision Applications18 Jun 2026
- 2026Brick-Composer: Using MLLMs for Assembly with Diverse Bricks3 Jun 2026
- 2026OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer15 Mar 2026
- 2026MixRI: Mixing Features of Reference Images for Novel Object Pose Estimation11 Jan 2026
- 2025Masked Registration and Autoencoding of CT Images for Predictive Tibia Reconstruction10 Dec 2025
- 2025SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation12 Nov 2025
- 2025Exploring Category-level Articulated Object Pose Tracking on SE(3) Manifolds8 Nov 2025
- 2025GS2POSE: Marry Gaussian Splatting to 6D Object Pose Estimation19 Oct 2025
- 2025SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams12 Oct 2025
- 2025Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimation5 Oct 2025
38 more
- 2025GAE: Unleashing Physical Potential of VLM with Generalizable Action Expert4 Oct 2025
- 2025Learning Point Cloud Representations with Pose Continuity for Depth-Based Category-Level 6D Object Pose Estimation20 Aug 2025
- 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulation7 Aug 2025
- 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training17 Jul 2025
- 2025Consensus-Driven Uncertainty for Robotic Grasping based on RGB Perception24 Jun 2025
- 2025DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation23 Jun 2025
- 2025Is Single-View Mesh Reconstruction Ready for Robotics?23 May 2025
- 2025DIMM: Decoupled Multi-hierarchy Kalman Filter for 3D Object Tracking18 May 2025
- 2025Active 6D Pose Estimation for Textureless Objects using Multi-View RGB Frames5 Mar 2025
- 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordance4 Dec 2024
- 20246DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting2 Dec 2024
- 2024Object Pose Estimation Using Implicit Representation For Transparent Objects17 Oct 2024
- 2024CVAM-Pose: Conditional Variational Autoencoder for Multi-Object Monocular Pose Estimation11 Oct 2024
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors7 Oct 2024
- 2024SSL-NBV: A Self-Supervised-Learning-Based Next-Best-View algorithm for Efficient 3D Plant Reconstruction by a Robot2 Oct 2024
- 2024FAFA: Frequency-Aware Flow-Aided Self-Supervision for Underwater Object Pose Estimation25 Sep 2024
- 2024LaPose: Laplacian Mixture Shape Modeling for RGB-Based Category-Level Object Pose Estimation24 Sep 2024
- 2024Good Grasps Only: A data engine for self-supervised fine-tuning of pose estimation using grasp poses for verification17 Sep 2024
- 2024Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation10 Sep 2024
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations8 Aug 2024
- 2024Adapting Skills to Novel Grasps: A Self-Supervised Approach31 Jul 2024
- 2024Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods31 Jul 2024
- 2024Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation13 Jun 2024
- 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation30 May 2024
- 2024Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation19 Mar 2024
- 2024GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splatting15 Mar 2024
- 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D22 Jan 2024
- 2023Learning Object State Changes in Videos: An Open-World Perspective19 Dec 2023
- 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation30 Nov 2023
- 2023Pseudo-keypoint RKHS Learning for Self-supervised 6DoF Pose Estimation16 Nov 2023
- 2023Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning6 Nov 2023
- 2023Key Point-based Orientation Estimation of Strawberries for Robotic Fruit Picking17 Oct 2023
- 2023MFOS: Model-Free & One-Shot Object Pose Estimation3 Oct 2023
- 2023DefGoalNet: Contextual Goal Learning from Demonstrations For Deformable Object Manipulation25 Sep 2023
- 2023Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors17 Sep 2023
- 2023DR-Pose: A Two-stage Deformation-and-Registration Pipeline for Category-level 6D Object Pose Estimation5 Sep 2023
- 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation30 Aug 2023
- 2023Leveraging Symmetries in Pick and Place15 Aug 2023
50 newest of 100 All 100 (JSON)
- 2023GenPose: Generative Category-level Object Pose Estimation via Diffusion Modelscited by 47
- 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordancecited by 35
- 2023ReorientDiff: Diffusion Model based Reorientation for Object Manipulationcited by 19
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priorscited by 5
- 2025Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimationcited by 1
- 2025DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulationcited by 0
- 2025Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimation5 Oct 2025
- 2025DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation23 Jun 2025
- 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordance4 Dec 2024
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors7 Oct 2024
- 2023GenPose: Generative Category-level Object Pose Estimation via Diffusion Models18 Jun 2023
- 2023ReorientDiff: Diffusion Model based Reorientation for Object Manipulation28 Feb 2023
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
- 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speedcited by 14
- 2024Imitation Learning-based Direct Visual Servoing using the Large Projection Formulationcited by 0
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations8 Aug 2024
- 2024Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation13 Jun 2024
- 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speed21 Jun 2022
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
- 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
- 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
- 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
- 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 54
- 2021SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimationcited by 53
- 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
- 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
- 2022Equivariant Transporter Networkcited by 44
12 most cited of 89 All 89 (JSON)
- 2026G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation20 Sep 2026
- 2026Learning to Throw Objects Safely in Multi-Obstacle Environments7 Jul 2026
- 2026Efficiently Linking Real Scenes with Synthetic Data Generation for AI-based Cognitive Robotics and Computer Vision Applications18 Jun 2026
- 2026Brick-Composer: Using MLLMs for Assembly with Diverse Bricks3 Jun 2026
- 2026OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer15 Mar 2026
- 2026MixRI: Mixing Features of Reference Images for Novel Object Pose Estimation11 Jan 2026
- 2025Masked Registration and Autoencoding of CT Images for Predictive Tibia Reconstruction10 Dec 2025
- 2025SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation12 Nov 2025
- 2025Exploring Category-level Articulated Object Pose Tracking on SE(3) Manifolds8 Nov 2025
- 2025GS2POSE: Marry Gaussian Splatting to 6D Object Pose Estimation19 Oct 2025
- 2025SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams12 Oct 2025
- 2025Learning Point Cloud Representations with Pose Continuity for Depth-Based Category-Level 6D Object Pose Estimation20 Aug 2025
12 newest of 89 All 89 (JSON)