# Reproducible Evaluation of Camera Auto-Exposure Methods in the Field: Platform, Benchmark and Lessons Learned

arXiv: https://arxiv.org/abs/2506.18844 · HTML: https://mlsanity.com/paper/2506.18844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Adaptive Risk Minimization: Learning to Adapt to Domain Shift](https://mlsanity.com/paper/2007.02931/) (2020)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models](https://mlsanity.com/paper/2205.01791/) (2022)
- [A Photometrically Calibrated Benchmark For Monocular Visual Odometry](https://mlsanity.com/paper/1607.02555/) (2016)
- [Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching](https://mlsanity.com/paper/2102.04341/) (2021)
- [Camera Exposure Control for Robust Robot Vision with Noise-Aware Image Quality Assessment](https://mlsanity.com/paper/1907.12646/) (2019)
- [Gradient-based Camera Exposure Control for Outdoor Mobile Platforms](https://mlsanity.com/paper/1708.07338/) (2017)
- [Online Photometric Calibration for Auto Exposure Video for Realtime Visual Odometry and SLAM](https://mlsanity.com/paper/1710.02081/) (2017)
- [Simultaneous Localization and Mapping Related Datasets: A Comprehensive Survey](https://mlsanity.com/paper/2102.04036/) (2021)
- [Targetless Calibration of LiDAR-IMU System Based on Continuous-time Batch Estimation](https://mlsanity.com/paper/2007.14759/) (2020)
- [Kilometer-scale autonomous navigation in subarctic forests: challenges and lessons learned](https://mlsanity.com/paper/2111.13981/) (2021)
- [Learning-based Image Enhancement for Visual Odometry in Challenging HDR Environments](https://mlsanity.com/paper/1707.01274/) (2017)
- [Exposing the Unseen: Exposure Time Emulation for Offline Benchmarking of Vision Algorithms](https://mlsanity.com/paper/2309.13139/) (2023)
- [Gravity-constrained point cloud registration](https://mlsanity.com/paper/2203.13799/) (2022)
- [PolyU-BPCoMa: A Dataset and Benchmark Towards Mobile Colorized Mapping Using a Backpack Multisensorial System](https://mlsanity.com/paper/2206.07468/) (2022)

## Built on by (most cited first, 1 of 1)

- [Adaptive Illumination Control for Robot Perception](https://mlsanity.com/paper/2602.15900/) (2026)

## Abstract (arXiv)

Standard datasets often present limitations, particularly due to the fixed nature of input data sensors, which makes it difficult to compare methods that actively adjust sensor parameters to suit environmental conditions. This is the case with Automatic-Exposure (AE) methods, which rely on environmental factors to influence the image acquisition process. As a result, AE methods have traditionally been benchmarked in an online manner, rendering experiments non-reproducible. Building on our prior work, we propose a methodology that utilizes an emulator capable of generating images at any exposure time. This approach leverages BorealHDR, a unique multi-exposure stereo dataset, along with its new extension, in which data was acquired along a repeated trajectory at different times of the day to assess the impact of changing illumination. In total, BorealHDR covers 13.4 km over 59 trajectories in challenging lighting conditions. The dataset also includes lidar-inertial-odometry-based maps with pose estimation for each image frame, as well as Global Navigation Satellite System (GNSS) data for comparison. We demonstrate that by using images acquired at various exposure times, we can emulate realistic images with a Root-Mean-Square Error (RMSE) below 1.78% compared to ground truth images. Using this offline approach, we benchmarked eight AE methods, concluding that the classical AE method remains the field's best performer. To further support reproducibility, we provide in-depth details on the development of our backpack acquisition platform, including hardware, electrical components, and performance specifications. Additionally, we share valuable lessons learned from deploying the backpack over more than 25 km across various environments. Our code and dataset are available online at this link: https://github.com/norlab-ulaval/TFR24 BorealHDR