# JVLDLoc: a Joint Optimization of Visual-LiDAR Constraints and Direction Priors for Localization in Driving Scenario

arXiv: https://arxiv.org/abs/2208.09777 · HTML: https://mlsanity.com/paper/2208.09777/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 21 Aug 2022.

## Builds on (most cited first, 16 of 16)

- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2104.10831/) (2021)
- [IMLS-SLAM: scan-to-model matching based on 3D data](https://mlsanity.com/paper/1802.08633/) (2018)
- [Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments](https://mlsanity.com/paper/2104.14938/) (2021)
- [Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences](https://mlsanity.com/paper/2004.00740/) (2020)
- [UV-SLAM: Unconstrained Line-based SLAM Using Vanishing Points for Structural Mapping](https://mlsanity.com/paper/2112.13515/) (2021)
- [LIMO: Lidar-Monocular Visual Odometry](https://mlsanity.com/paper/1807.07524/) (2018)
- [R2LIVE: A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mapping](https://mlsanity.com/paper/2102.12400/) (2021)
- [LiDAR Enhanced Structure-from-Motion](https://mlsanity.com/paper/1911.03369/) (2019)
- [CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description](https://mlsanity.com/paper/2001.01354/) (2020)
- [ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System](https://mlsanity.com/paper/1909.12947/) (2019)

## Abstract (arXiv)

The ability for a moving agent to localize itself in environment is the basic demand for emerging applications, such as autonomous driving, etc. Many existing methods based on multiple sensors still suffer from drift. We propose a scheme that fuses map prior and vanishing points from images, which can establish an energy term that is only constrained on rotation, called the direction projection error. Then we embed these direction priors into a visual-LiDAR SLAM system that integrates camera and LiDAR measurements in a tightly-coupled way at backend. Specifically, our method generates visual reprojection error and point to Implicit Moving Least Square(IMLS) surface of scan constraints, and solves them jointly along with direction projection error at global optimization. Experiments on KITTI, KITTI-360 and Oxford Radar Robotcar show that we achieve lower localization error or Absolute Pose Error (APE) than prior map, which validates our method is effective.