mlsanityData to 7 Oct 2026

UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering

arXiv 2102.11870 ↗First posted 23 Feb 2021cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

Aligning partial views of a scene into a single whole is essential to understanding one's environment and is a key component of numerous robotics tasks such as SLAM and SfM. Recent approaches have proposed end-to-end systems that can outperform traditional methods by leveraging pose supervision. However, with the rising prevalence of cameras with depth sensors, we can expect a new stream of raw RGB-D data without the annotations needed for supervision. We propose UnsupervisedR&R: an end-to-end unsupervised approach to learning point cloud registration from raw RGB-D video. The key idea is to leverage differentiable alignment and rendering to enforce photometric and geometric consistency between frames. We evaluate our approach on indoor scene datasets and find that we outperform existing traditional approaches with classic and learned descriptors while being competitive with supervised geometric point cloud registration approaches.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 55 of its 81 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

55 of 81 references.

References · 55

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  4. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  5. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  6. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  8. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  10. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  11. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  12. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
38 more
  1. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  2. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  3. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  4. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  5. 2019Deep Closest Point: Learning Representations for Point Cloud Registrationcited by 318
  6. 2016LIFT: Learned Invariant Feature Transformcited by 305
  7. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 264
  8. 2019PointNetLK: Robust & Efficient Point Cloud Registration using PointNetcited by 231
  9. 2020Deep Global Registrationcited by 228
  10. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  11. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  12. 2017Learning a Multi-View Stereo Machinecited by 194
  13. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  14. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  15. 2020RPM-Net: Robust Point Matching using Learned Featurescited by 181
  16. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  17. 2017Learning to Find Good Correspondencescited by 176
  18. 2020D3Feat: Joint Learning of Dense Detection and Description of 3D Local Featurescited by 175
  19. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  20. 2018The Perfect Match: 3D Point Cloud Matching with Smoothed Densitiescited by 169
  21. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  22. 2019PRNet: Self-Supervised Learning for Partial-to-Partial Registrationcited by 146
  23. 2016Learning-Based View Synthesis for Light Field Camerascited by 145
  24. 2018PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptorscited by 140
  25. 2019Neural-Guided RANSAC: Learning Where to Sample Model Hypothesescited by 129
  26. 20193D Ken Burns Effect from a Single Imagecited by 128
  27. 2015Learning to Compare Image Patches via Convolutional Neural Networkscited by 109
  28. 2020Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondencescited by 108
  29. 20183DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registrationcited by 100
  30. 2020DeepGMR: Learning Latent Gaussian Mixture Models for Registrationcited by 91
  31. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  32. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  33. 2020Articulation-aware Canonical Surface Mappingcited by 80
  34. 2020Learning multiview 3D point cloud registrationcited by 78
  35. 2019DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registrationcited by 77
  36. 2017Learning to Synthesize a 4D RGBD Light Field from a Single Imagecited by 60
  37. 2014Viewpoints and Keypointscited by 59
  38. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 10; 2024: 12; 2025: 5; 2026: 1.

44 papers.

Sort:

  1. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  2. 2021Lepard: Learning partial point cloud matching in rigid and deformable scenescited by 85
  3. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  4. 2023SiLK -- Simple Learned Keypointscited by 54
  5. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  6. 2021Planar Surface Reconstruction from Sparse Viewscited by 40
  7. 2022The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTscited by 38
  8. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  9. 2021Bootstrap Your Own Correspondencescited by 27
  10. 2021R-PointHop: A Green, Accurate, and Unsupervised Point Cloud Registration Methodcited by 26
  11. 2022PlaneFormers: From Sparse View Planes to 3D Reconstructioncited by 26
  12. 2023Unsupervised Deep Probabilistic Approach for Partial Point Cloud Registrationcited by 26
32 more
  1. 2022Last-Mile Embodied Visual Navigationcited by 25
  2. 2022iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Trackingcited by 22
  3. 2022Reliable Inlier Evaluation for Unsupervised Point Cloud Registrationcited by 19
  4. 2023Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotationcited by 18
  5. 2024NeRFs in Robotics: A Surveycited by 16
  6. 2024Extend Your Own Correspondences: Unsupervised Distant Point Cloud Registration by Progressive Distance Extensioncited by 15
  7. 2023PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registrationcited by 14
  8. 2023PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometrycited by 13
  9. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  10. 2022Improving RGB-D Point Cloud Registration by Learning Multi-scale Local Linear Transformationcited by 12
  11. 2022Towards Self-Supervised Category-Level Object Pose and Size Estimationcited by 11
  12. 2026CMHANet: A Cross-Modal Hybrid Attention Network for Point Cloud Registrationcited by 9
  13. 2025From Thousands to Billions: 3D Visual Language Grounding via Render-Supervised Distillation from 2D VLMscited by 7
  14. 2023ZeroReg: Zero-Shot Point Cloud Registration with Foundation Modelscited by 6
  15. 2021COTReg:Coupled Optimal Transport based Point Cloud Registrationcited by 5
  16. 2022Self-Supervised Correspondence Estimation via Multiview Registrationcited by 5
  17. 2023RoCNet: 3D Robust Registration of Point-Clouds using Deep Learningcited by 2
  18. 2023Overlap Bias Matching is Necessary for Point Cloud Registrationcited by 2
  19. 2024RGBD-Glue: General Feature Combination for Robust RGB-D Point Cloud Registrationcited by 2
  20. 2024Equi-GSPR: Equivariant SE(3) Graph Network Model for Sparse Point Cloud Registrationcited by 2
  21. 20243D Registration in 30 Years: A Surveycited by 2
  22. 2024CSCPR: Cross-Source-Context Indoor RGB-D Place Recognitioncited by 1
  23. 2024Mining and Transferring Feature-Geometry Coherence for Unsupervised Point Cloud Registrationcited by 1
  24. 2022Accurate Ground-Truth Depth Image Generation via Overfit Training of Point Cloud Registration using Local Frame Setscited by 0
  25. 2023One-Nearest Neighborhood Guides Inlier Estimation for Unsupervised Point Cloud Registrationcited by 0
  26. 2024NeRF-Supervised Feature Point Detection and Descriptioncited by 0
  27. 2024F2M-Reg: Unsupervised RGB-D Point Cloud Registration with Frame-to-Model Optimizationcited by 0
  28. 2024MPVO: Motion-Prior based Visual Odometry for PointGoal Navigationcited by 0
  29. 2025DINOReg: Strong Point Cloud Registration with Vision Foundation Modelcited by 0
  30. 2025Leveraging Cycle-Consistent Anchor Points for Self-Supervised RGB-D Registrationcited by 0
  31. 2025ViGG: Robust RGB-D Point Cloud Registration using Visual-Geometric Mutual Guidancecited by 0
  32. 2025Geometry-to-Image Synthesis-Driven Generative Point Cloud Registrationcited by 0