# Hold-Out Self-Validation Cannot Certify Photogrammetric Accuracy: Saturation and Blindness to Coherent Distortion

arXiv: https://arxiv.org/abs/2607.24852 · HTML: https://mlsanity.com/paper/2607.24852/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [RayZer: A Self-supervised Large View Synthesis Model](https://mlsanity.com/paper/2505.00702/) (2025)
- [The Hessigheim 3D (H3D) Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Textured Meshes from UAV LiDAR and Multi-View-Stereo](https://mlsanity.com/paper/2102.05346/) (2021)
- [E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training](https://mlsanity.com/paper/2512.10950/) (2025)
- [Look Ma, No Ground Truth! Ground-Truth-Free Tuning of Structure from Motion and Visual SLAM](https://mlsanity.com/paper/2412.01116/) (2024)

## Abstract (arXiv)

Internal self-consistency cannot certify the accuracy of a photogrammetric reconstruction, and the failure is structural rather than a matter of tuning. This matters because hold-out self-validation scores are increasingly offered as quality evidence for metric deliverables whose correctness is otherwise unknown without an external survey. We formalise a track-leakage-free hold-out protocol: a deterministic image subset is withheld, and each withheld view is re-localised against only those 3D points supported by two or more retained images, so no view is tested against structure it helped create. We evaluate it on five GNSS-referenced captures across four sites, 13 ETH3D scenes, a EuRoC flight and 30 IMC 2025 scenes. The protocol is well-posed but does not measure accuracy. It saturates: the internal confidence score stays pinned at 1.00 while true error swings 14.1x within one capture. It is blind to coherent distortion: fragmenting corruption is caught, but internally self-consistent, globally distorted models are not, and were wrong by 55-106 m at confidence 1.00 at three of four captures. On IMC 2025 it separates failed from successful reconstructions (rho = 0.68) yet ranks nothing among the successful (rho = 0.01). Track-leakage-free hold-out measures internal geometric consistency: a fragmentation warning, not a substitute for control-point accuracy assessment.