mlsanityData to 14 Oct 2026

Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms

arXiv 2105.14246 ↗First posted 29 May 2021cs.RO cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Orienting objects is a critical component in the automation of many packing and assembly tasks. We present an algorithm to orient novel objects given a depth image of the object in its current and desired orientation. We formulate a self-supervised objective for this problem and train a deep neural network to estimate the 3D rotation as parameterized by a quaternion, between these current and desired depth images. We then use the trained network in a proportional controller to re-orient objects based on the estimated rotation between the two depth images. Results suggest that in simulation we can rotate unseen objects with unknown geometries by up to 30deg with a median angle error of 1.47deg over 100 random initial/desired orientations each for 22 novel objects. Experiments on physical objects suggest that the controller can achieve a median angle error of 4.2deg over 10 random initial/desired orientations each for 5 objects.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 29 May 2021.

Builds on

18 of 31 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  4. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  5. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  6. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 101
  7. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 89
  8. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  9. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  10. 2020EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulationcited by 61
  11. 2017Relative Camera Pose Estimation Using Convolutional Neural Networkscited by 47
  12. 2018Deep Directional Statistics: Pose Estimation with Uncertainty Quantificationcited by 45
6 more
  1. 2019Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objectscited by 39
  2. 2019Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Cluttercited by 30
  3. 2020A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertaintycited by 30
  4. 2018Deep-6DPose: Recovering 6D Object Pose from a Single RGB Imagecited by 26
  5. 2020Robust 6D Object Pose Estimation by Learning RGB-D Featurescited by 25
  6. 2018Pose Estimation for Objects with Rotational Symmetrycited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 3; 2024: 8; 2025: 1; 2026: 1.

17 papers.

Sort:

By area:

  1. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  2. 2022TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulationcited by 47
  3. 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
  4. 2023DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classificationcited by 23
  5. 2023APLA: Additional Perturbation for Latent Noise with Adversarial Training Enables Consistencycited by 9
  6. 2024CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimatorscited by 5
  7. 2021Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavitiescited by 4
  8. 2024Learning Correspondence for Deformable Objectscited by 2
  9. 2024Safe Deep Model-Based Reinforcement Learning with Lyapunov Functionscited by 2
  10. 2024CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertaintycited by 2
  11. 2022Self-supervised Wide Baseline Visual Servoing via 3D Equivariancecited by 1
  12. 2025H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flowscited by 1
5 more
  1. 2024Towards Improving Learning from Demonstration Algorithms via MCMC Methodscited by 0
  2. 2024Theoretical Analysis for Expectation-Maximization-Based Multi-Model 3D Registrationcited by 0
  3. 2024Rocket Landing Control with Grid Fins and Path-following using MPCcited by 0
  4. 2024Adapting Skills to Novel Grasps: A Self-Supervised Approachcited by 0
  5. 2026Self-Supervised Visual Representation Learning: Pretrain-Finetuning or Joint Training?cited by 0