Differentiable SLAM Helps Deep Learning-based LiDAR Perception Tasks
Not on the map. Why?
Abstract (arXiv)
We investigate a new paradigm that uses differentiable SLAM architectures in a self-supervised manner to train end-to-end deep learning models in various LiDAR based applications. To the best of our knowledge there does not exist any work that leverages SLAM as a training signal for deep learning based models. We explore new ways to improve the efficiency, robustness, and adaptability of LiDAR systems with deep learning techniques. We focus on the potential benefits of differentiable SLAM architectures for improving performance of deep learning tasks such as classification, regression as well as SLAM. Our experimental results demonstrate a non-trivial increase in the performance of two deep learning applications - Ground Level Estimation and Dynamic to Static LiDAR Translation, when used with differentiable SLAM architectures. Overall, our findings provide important insights that enhance the performance of LiDAR based navigation systems. We demonstrate that this new paradigm of using SLAM Loss signal while training LiDAR based models can be easily adopted by the community.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 17 Sep 2023.
Builds on
31 of 45 references.
References · 31
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
- 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
- 2020Center-based 3D Object Detection and Trackingcited by 1,144
- 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
- 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
- 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
- 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
- 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
- 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
- 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
- 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
19 more
- 2017CNN-SLAM: Real-time dense monocular SLAM with learned depth predictioncited by 152
- 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 152
- 2022Learning to Generate Realistic LiDAR Point Cloudscited by 93
- 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
- 2018Deep Generative Modeling of LiDAR Datacited by 71
- 2021Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Datacited by 64
- 2022Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloudcited by 47
- 2019SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representationscited by 37
- 2021Patchwork: Concentric Zone-based Region-wise Ground Segmentation with Ground Likelihood Estimation Using a 3D LiDAR Sensorcited by 29
- 2022Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimationcited by 28
- 2019gradSLAM: Automagically differentiable SLAMcited by 27
- 2021Learning to Drop Points for LiDAR Scan Synthesiscited by 27
- 2022Generative Range Imaging for Learning Scene Priors of 3D LiDAR Datacited by 18
- 2021Differentiable Factor Graph Optimization for Learning Smootherscited by 17
- 2022Learning to Simulate Realistic LiDARscited by 16
- 2021LEO: Learning Energy-based Models in Factor Graph Optimizationcited by 8
- 2021Automatic labelling of urban point clouds using data fusioncited by 6
- 2022SLAM-Supported Self-Training for 6D Object Pose Estimationcited by 6
- 2022A Realism Metric for Generated LiDAR Point Cloudscited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.