mlsanityData to 14 Oct 2026

Self-Supervised Correspondence Estimation via Multiview Registration

arXiv 2212.03236 ↗First posted 6 Dec 2022cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformation synchronization algorithm. Our key insight is that self-supervised multiview registration allows us to obtain correspondences over longer time frames; increasing both the diversity and difficulty of sampled pairs. We evaluate our approach on indoor scenes for correspondence estimation and RGB-D pointcloud registration and find that we perform on-par with supervised approaches.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 47 of its 81 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

47 of 81 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2017Billion-scale similarity search with GPUscited by 2,497
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  8. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  9. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  10. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  11. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
35 more
  1. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  2. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  3. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 336
  4. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 290
  5. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 266
  6. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  7. 2020Deep Global Registrationcited by 229
  8. 2020RPM-Net: Robust Point Matching using Learned Featurescited by 182
  9. 2017Learning to Find Good Correspondencescited by 176
  10. 2018Neighbourhood Consensus Networkscited by 174
  11. 2018The Perfect Match: 3D Point Cloud Matching with Smoothed Densitiescited by 169
  12. 2019Does Object Recognition Work for Everyone?cited by 151
  13. 2017Convolutional neural network architecture for geometric matchingcited by 138
  14. 2019Neural-Guided RANSAC: Learning Where to Sample Model Hypothesescited by 129
  15. 2019Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filterscited by 118
  16. 2016BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integrationcited by 116
  17. 2020Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondencescited by 109
  18. 2020Kernel Operations on the GPU, with Autodiff, without Memory Overflowscited by 104
  19. 2020Learning multiview 3D point cloud registrationcited by 78
  20. 2019DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registrationcited by 77
  21. 2015Sparse 3D convolutional neural networkscited by 72
  22. 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
  23. 2020PointGMM: a Neural GMM Network for Point Cloudscited by 33
  24. 2019Learning Transformation Synchronizationcited by 29
  25. 2021Bootstrap Your Own Correspondencescited by 27
  26. 2019SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondencescited by 26
  27. 20193D Local Features for Direct Pairwise Registrationcited by 26
  28. 2021Self-supervised Geometric Perceptioncited by 26
  29. 2016Learning Covariant Feature Detectorscited by 25
  30. 2019Self-supervised Learning for Video Correspondence Flowcited by 21
  31. 2020Synchronizing Probability Measures on Rotations via Optimal Transportcited by 19
  32. 2022End2End Multi-View Feature Matching with Differentiable Pose Optimizationcited by 19
  33. 2022Improving RGB-D Point Cloud Registration by Learning Multi-scale Local Linear Transformationcited by 12
  34. 2021Effect of Parameter Optimization on Classical and Learning-based Image Matching Methodscited by 5
  35. 2021Digging Into Self-Supervised Learning of Feature Descriptorscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2024PF3plat: Pose-Free Feed-Forward 3D Gaussian Splattingcited by 62
  2. 2023Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweightingcited by 19
  3. 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6
  4. 2024Geometry Depth Consistency in RGBD Relative Pose Estimationcited by 0
  5. 2025Leveraging Cycle-Consistent Anchor Points for Self-Supervised RGB-D Registrationcited by 0