# Pseudo-LiDAR for Visual Odometry arXiv: https://arxiv.org/abs/2209.01567 ยท HTML: https://mlsanity.com/paper/2209.01567/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 42 of its 60 references that are in the map. - First posted on arXiv 4 Sep 2022. ## Builds on (most cited first, 30 of 42) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016) - [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017) - [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018) - [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018) - [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019) - [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019) - [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017) - [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018) - [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019) - [DeepV2D: Video to Depth with Differentiable Structure from Motion](https://mlsanity.com/paper/1812.04605/) (2018) - [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018) - [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017) - [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019) - [EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection](https://mlsanity.com/paper/2007.08856/) (2020) - [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018) - [Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud](https://mlsanity.com/paper/1903.09847/) (2019) - [TartanVO: A Generalizable Learning-based VO](https://mlsanity.com/paper/2011.00359/) (2020) - [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017) - [Deep Auxiliary Learning for Visual Localization and Odometry](https://mlsanity.com/paper/1803.03642/) (2018) - [EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2106.15277/) (2021) - [Visual Odometry Revisited: What Should Be Learnt?](https://mlsanity.com/paper/1909.09803/) (2019) - [Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth](https://mlsanity.com/paper/1904.07087/) (2019) - [Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction](https://mlsanity.com/paper/2004.10681/) (2020) - [Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling](https://mlsanity.com/paper/2007.10983/) (2020) - [PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization](https://mlsanity.com/paper/2012.00972/) (2020) - [Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry](https://mlsanity.com/paper/1904.01892/) (2019) - [Efficient 3D Deep LiDAR Odometry](https://mlsanity.com/paper/2111.02135/) (2021) - [Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry](https://mlsanity.com/paper/2003.00766/) (2020) - [Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry](https://mlsanity.com/paper/1908.08704/) (2019) ## Built on by (most cited first, 2 of 2) - [NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation](https://mlsanity.com/paper/2405.14241/) (2024) - [An Online Adaptation Method for Robust Depth Estimation and Visual Odometry in the Open World](https://mlsanity.com/paper/2504.11698/) (2025) ## Abstract (arXiv) In the existing methods, LiDAR odometry shows superior performance, but visual odometry is still widely used for its price advantage. Conventionally, the task of visual odometry mainly rely on the input of continuous images. However, it is very complicated for the odometry network to learn the epipolar geometry information provided by the images. In this paper, the concept of pseudo-LiDAR is introduced into the odometry to solve this problem. The pseudo-LiDAR point cloud back-projects the depth map generated by the image into the 3D point cloud, which changes the way of image representation. Compared with the stereo images, the pseudo-LiDAR point cloud generated by the stereo matching network can get the explicit 3D coordinates. Since the 6 Degrees of Freedom (DoF) pose transformation occurs in 3D space, the 3D structure information provided by the pseudo-LiDAR point cloud is more direct than the image. Compared with sparse LiDAR, the pseudo-LiDAR has a denser point cloud. In order to make full use of the rich point cloud information provided by the pseudo-LiDAR, a projection-aware dense odometry pipeline is adopted. Most previous LiDAR-based algorithms sampled 8192 points from the point cloud as input to the odometry network. The projection-aware dense odometry pipeline takes all the pseudo-LiDAR point clouds generated from the images except for the error points as the input to the network. While making full use of the 3D geometric information in the images, the semantic information in the images is also used in the odometry task. The fusion of 2D-3D is achieved in an image-only based odometry. Experiments on the KITTI dataset prove the effectiveness of our method. To the best of our knowledge, this is the first visual odometry method using pseudo-LiDAR.