Pseudo-LiDAR for Visual Odometry
Not on the map. Why?
Abstract (arXiv)
In the existing methods, LiDAR odometry shows superior performance, but visual odometry is still widely used for its price advantage. Conventionally, the task of visual odometry mainly rely on the input of continuous images. However, it is very complicated for the odometry network to learn the epipolar geometry information provided by the images. In this paper, the concept of pseudo-LiDAR is introduced into the odometry to solve this problem. The pseudo-LiDAR point cloud back-projects the depth map generated by the image into the 3D point cloud, which changes the way of image representation. Compared with the stereo images, the pseudo-LiDAR point cloud generated by the stereo matching network can get the explicit 3D coordinates. Since the 6 Degrees of Freedom (DoF) pose transformation occurs in 3D space, the 3D structure information provided by the pseudo-LiDAR point cloud is more direct than the image. Compared with sparse LiDAR, the pseudo-LiDAR has a denser point cloud. In order to make full use of the rich point cloud information provided by the pseudo-LiDAR, a projection-aware dense odometry pipeline is adopted. Most previous LiDAR-based algorithms sampled 8192 points from the point cloud as input to the odometry network. The projection-aware dense odometry pipeline takes all the pseudo-LiDAR point clouds generated from the images except for the error points as the input to the network. While making full use of the 3D geometric information in the images, the semantic information in the images is also used in the odometry task. The fusion of 2D-3D is achieved in an image-only based odometry. Experiments on the KITTI dataset prove the effectiveness of our method. To the best of our knowledge, this is the first visual odometry method using pseudo-LiDAR.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 4 Sep 2022.
Builds on
42 of 60 references.
References · 42
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
- 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
- 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
- 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 244
- 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
- 2017Geometric Loss Functions for Camera Pose Regression with Deep Learningcited by 224
- 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
- 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
- 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
- 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
30 more
- 2017DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networkscited by 175
- 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
- 2020EPNet: Enhancing Point Features with Image Semantics for 3D Object Detectioncited by 164
- 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
- 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
- 2020TartanVO: A Generalizable Learning-based VOcited by 108
- 2017UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learningcited by 95
- 2018Deep Auxiliary Learning for Visual Localization and Odometrycited by 59
- 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
- 2019Visual Odometry Revisited: What Should Be Learnt?cited by 57
- 2019Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depthcited by 50
- 2020Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Predictioncited by 41
- 2020Learning Monocular Visual Odometry via Self-Supervised Long-Term Modelingcited by 39
- 2020PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimizationcited by 35
- 2019Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometrycited by 27
- 2021Efficient 3D Deep LiDAR Odometrycited by 25
- 2020Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometrycited by 21
- 2019Sequential Adversarial Learning for Self-Supervised Deep Visual Odometrycited by 18
- 2022Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selectioncited by 18
- 2019Beyond Photometric Loss for Self-Supervised Ego-Motion Estimationcited by 17
- 2019Pose Graph Optimization for Unsupervised Monocular Visual Odometrycited by 17
- 2021Residual 3D Scene Flow Learning with Context-Aware Feature Extractioncited by 15
- 2018Guided Feature Selection for Deep Visual Odometrycited by 10
- 2021Deep Online Correction for Monocular Visual Odometrycited by 9
- 2020Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervisioncited by 8
- 2022Unsupervised Learning of 3D Scene Flow from Monocular Cameracited by 7
- 2021Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Maskscited by 6
- 2021DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometrycited by 6
- 2018Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotioncited by 5
- 2021Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustmentcited by 5
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.