# Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera System

arXiv: https://arxiv.org/abs/1809.05477 · HTML: https://mlsanity.com/paper/1809.05477/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2018.

## Built on by (most cited first, 30 of 34)

- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines](https://mlsanity.com/paper/2103.05086/) (2021)
- [FocusTune: Tuning Visual Localization through Focus-Guided Sampling](https://mlsanity.com/paper/2311.02872/) (2023)
- [GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization](https://mlsanity.com/paper/2409.16502/) (2024)
- [Affine Correspondences between Multi-Camera Systems for Relative Pose Estimation](https://mlsanity.com/paper/2306.12996/) (2023)
- [Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors](https://mlsanity.com/paper/2203.03334/) (2022)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [City-scale Scene Change Detection using Point Clouds](https://mlsanity.com/paper/2103.14314/) (2021)
- [Privacy-Preserving Representations are not Enough -- Recovering Scene Content from Camera Poses](https://mlsanity.com/paper/2305.04603/) (2023)
- [Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement](https://mlsanity.com/paper/2406.08463/) (2024)
- [GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations](https://mlsanity.com/paper/2508.18242/) (2025)
- [A Critical Analysis of Internal Reliability for Uncertainty Quantification of Dense Image Matching in Multi-view Stereo](https://mlsanity.com/paper/2309.09379/) (2023)
- [Combining Absolute and Semi-Generalized Relative Poses for Visual Localization](https://mlsanity.com/paper/2409.14269/) (2024)
- [Accurate calibration of multi-perspective cameras from a generalization of the hand-eye constraint](https://mlsanity.com/paper/2202.00886/) (2022)
- [Scalable and Generalizable Correspondence Pruning via Geometry-Consistent Pre-training](https://mlsanity.com/paper/2406.05773/) (2024)
- [Constrained Bundle Adjustment for Structure From Motion Using Uncalibrated Multi-Camera Systems](https://mlsanity.com/paper/2204.04145/) (2022)
- [Semantic keypoint extraction for scanned animals using multi-depth-camera systems](https://mlsanity.com/paper/2211.08634/) (2022)
- [Globally Optimal Solution to the Generalized Relative Pose Estimation Problem using Affine Correspondences](https://mlsanity.com/paper/2512.17188/) (2025)
- [Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos](https://mlsanity.com/paper/2603.12064/) (2026)
- [A Pose-only Geometric Constraint for Multi-Camera Pose Adjustment](https://mlsanity.com/paper/2604.23704/) (2026)
- [Object Detection During Newborn Resuscitation Activities](https://mlsanity.com/paper/2303.07790/) (2023)
- [Robust Localization with Visual-Inertial Odometry Constraints for Markerless Mobile AR](https://mlsanity.com/paper/2308.05394/) (2023)
- [Relative Pose for Nonrigid Multi-Perspective Cameras: The Static Case](https://mlsanity.com/paper/2401.09140/) (2024)

## Abstract (arXiv)

Project AutoVision aims to develop localization and 3D scene perception capabilities for a self-driving vehicle. Such capabilities will enable autonomous navigation in urban and rural environments, in day and night, and with cameras as the only exteroceptive sensors. The sensor suite employs many cameras for both 360-degree coverage and accurate multi-view stereo; the use of low-cost cameras keeps the cost of this sensor suite to a minimum. In addition, the project seeks to extend the operating envelope to include GNSS-less conditions which are typical for environments with tall buildings, foliage, and tunnels. Emphasis is placed on leveraging multi-view geometry and deep learning to enable the vehicle to localize and perceive in 3D space. This paper presents an overview of the project, and describes the sensor suite and current progress in the areas of calibration, localization, and perception.