# OmniColor: A Global Camera Pose Optimization Approach of LiDAR-360Camera Fusion for Colorizing Point Clouds

arXiv: https://arxiv.org/abs/2404.04693 · HTML: https://mlsanity.com/paper/2404.04693/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [OpenVSLAM: A Versatile Visual SLAM Framework](https://mlsanity.com/paper/1910.01122/) (2019)
- [R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package](https://mlsanity.com/paper/2109.07982/) (2021)
- [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021)
- [LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2104.10831/) (2021)
- [Large-scale Localization Datasets in Crowded Indoor Spaces](https://mlsanity.com/paper/2105.08941/) (2021)
- [BALM: Bundle Adjustment for Lidar Mapping](https://mlsanity.com/paper/2010.08215/) (2020)
- [FAST-LIVO: Fast and Tightly-coupled Sparse-Direct LiDAR-Inertial-Visual Odometry](https://mlsanity.com/paper/2203.00893/) (2022)
- [Efficient and Probabilistic Adaptive Voxel Mapping for Accurate Online LiDAR Odometry](https://mlsanity.com/paper/2109.07082/) (2021)
- [PICCOLO: Point Cloud-Centric Omnidirectional Localization](https://mlsanity.com/paper/2108.06545/) (2021)
- [Efficient and Consistent Bundle Adjustment on Lidar Point Clouds](https://mlsanity.com/paper/2209.08854/) (2022)
- [Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras using Adaptive Voxelization](https://mlsanity.com/paper/2109.06550/) (2021)
- [A Joint Optimization Approach of LiDAR-Camera Fusion for Accurate Dense 3D Reconstructions](https://mlsanity.com/paper/1907.00930/) (2019)
- [R3LIVE++: A Robust, Real-time, Radiance reconstruction package with a tightly-coupled LiDAR-Inertial-Visual state Estimator](https://mlsanity.com/paper/2209.03666/) (2022)
- [Coarse-to-fine Hybrid 3D Mapping System with Co-calibrated Omnidirectional Camera and Non-repetitive LiDAR](https://mlsanity.com/paper/2301.12934/) (2023)
- [CP+: Camera Poses Augmentation with Large-scale LiDAR Maps](https://mlsanity.com/paper/2302.12198/) (2023)

## Built on by (most cited first, 5 of 5)

- [3D Gaussian Inverse Rendering with Approximated Global Illumination](https://mlsanity.com/paper/2504.01358/) (2025)
- [FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera](https://mlsanity.com/paper/2409.15054/) (2024)
- [Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI](https://mlsanity.com/paper/2511.20620/) (2025)
- [Decision-Driven Semantic Object Exploration for Legged Robots via Confidence-Calibrated Perception and Topological Subgoal Selection](https://mlsanity.com/paper/2509.20739/) (2025)
- [LiDAR Point Cloud Colourisation Using Multi-Camera Fusion and Low-Light Image Enhancement](https://mlsanity.com/paper/2509.25859/) (2025)

## Abstract (arXiv)

A Colored point cloud, as a simple and efficient 3D representation, has many advantages in various fields, including robotic navigation and scene reconstruction. This representation is now commonly used in 3D reconstruction tasks relying on cameras and LiDARs. However, fusing data from these two types of sensors is poorly performed in many existing frameworks, leading to unsatisfactory mapping results, mainly due to inaccurate camera poses. This paper presents OmniColor, a novel and efficient algorithm to colorize point clouds using an independent 360-degree camera. Given a LiDAR-based point cloud and a sequence of panorama images with initial coarse camera poses, our objective is to jointly optimize the poses of all frames for mapping images onto geometric reconstructions. Our pipeline works in an off-the-shelf manner that does not require any feature extraction or matching process. Instead, we find optimal poses by directly maximizing the photometric consistency of LiDAR maps. In experiments, we show that our method can overcome the severe visual distortion of omnidirectional images and greatly benefit from the wide field of view (FOV) of 360-degree cameras to reconstruct various scenarios with accuracy and stability. The code will be released at https://github.com/liubonan123/OmniColor/.