mlsanityData to 13 Oct 2026

Self-supervised Visual-LiDAR Odometry with Flip Consistency

arXiv 2101.01322 ↗First posted 5 Jan 2021cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Most learning-based methods estimate ego-motion by utilizing visual sensors, which suffer from dramatic lighting variations and textureless scenarios. In this paper, we incorporate sparse but accurate depth measurements obtained from lidars to overcome the limitation of visual methods. To this end, we design a self-supervised visual-lidar odometry (Self-VLO) framework. It takes both monocular images and sparse depth maps projected from 3D lidar points as input, and produces pose and depth estimations in an end-to-end learning manner, without using any ground truth labels. To effectively fuse two modalities, we design a two-pathway encoder to extract features from visual and depth images and fuse the encoded features with those in decoders at multiple scales by our fusion module. We also adopt a siamese architecture and design an adaptively weighted flip consistency loss to facilitate the self-supervised learning of our VLO. Experiments on the KITTI odometry benchmark show that the proposed approach outperforms all self-supervised visual or lidar odometries. It also performs better than fully supervised VOs, demonstrating the power of fusion.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 5 Jan 2021.

Builds on

35 of 54 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,991
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,269
  6. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  7. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 803
  8. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 779
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  11. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
  12. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
23 more
  1. 2017DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networkscited by 173
  2. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  3. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  4. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151
  5. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  6. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  7. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  8. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  9. 2017UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learningcited by 95
  10. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  11. 2019DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registrationcited by 77
  12. 2019LO-Net: Deep Real-time Lidar Odometrycited by 59
  13. 2019Bilateral Cyclic Constraint and Adaptive Regularization for Unsupervised Monocular Depth Predictioncited by 44
  14. 2020Self-Supervised Deep Visual Odometry with Online Adaptationcited by 34
  15. 2019Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometrycited by 27
  16. 2019Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distancescited by 27
  17. 2018LIMO: Lidar-Monocular Visual Odometrycited by 21
  18. 2019Sequential Adversarial Learning for Self-Supervised Deep Visual Odometrycited by 18
  19. 2020LOL: Lidar-Only Odometry and Localization in 3D Point Cloud Mapscited by 13
  20. 2018Guided Feature Selection for Deep Visual Odometrycited by 10
  21. 2019Learn Stereo, Infer Mono: Siamese Networks for Self-Supervised, Monocular, Depth Estimationcited by 6
  22. 2018Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotioncited by 5
  23. 2020LiDAR Data Enrichment Using Deep Learning Based on High-Resolution Image: An Approach to Achieve High-Performance LiDAR SLAM Using Low-cost LiDARcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 1; 2025: 2.

9 papers.

  1. 2024DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignmentcited by 25
  2. 2023HypLiLoc: Towards Effective LiDAR Pose Regression with Hyperbolic Fusioncited by 24
  3. 20234DRVO-Net: Deep 4D Radar-Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusioncited by 11
  4. 2022Self-Supervised Ego-Motion Estimation Based on Multi-Layer Fusion of RGB and Inferred Depthcited by 6
  5. 2021What's in My LiDAR Odometry Toolbox?cited by 4
  6. 2022MoDA: Map style transfer for self-supervised Domain Adaptation of embodied agentscited by 3
  7. 2025DVLO4D: Deep Visual-Lidar Odometry with Sparse Spatial-temporal Fusioncited by 2
  8. 2023Calibrating Panoramic Depth Estimation for Practical Localization and Mappingcited by 1
  9. 2025Dense-depth map guided deep Lidar-Visual Odometry with Sparse Point Clouds and Imagescited by 0