# GeoCalib: Learning Single-image Calibration with Geometric Optimization

arXiv: https://arxiv.org/abs/2409.06704 · HTML: https://mlsanity.com/paper/2409.06704/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 91 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation](https://mlsanity.com/paper/2209.08575/) (2022)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Perspective Fields for Single Image Camera Calibration](https://mlsanity.com/paper/2212.03239/) (2022)
- [End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization](https://mlsanity.com/paper/1909.06043/) (2019)
- [Theseus: A Library for Differentiable Nonlinear Optimization](https://mlsanity.com/paper/2207.09442/) (2022)
- [DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients](https://mlsanity.com/paper/2212.07766/) (2022)
- [The Double Sphere Camera Model](https://mlsanity.com/paper/1807.08957/) (2018)
- [Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task](https://mlsanity.com/paper/1912.00623/) (2019)
- [Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimization](https://mlsanity.com/paper/2007.14628/) (2020)
- [CTRL-C: Camera calibration TRansformer with Line-Classification](https://mlsanity.com/paper/2109.02259/) (2021)
- [CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus](https://mlsanity.com/paper/2001.02643/) (2020)
- [Generalized Differentiable RANSAC](https://mlsanity.com/paper/2212.13185/) (2022)
- [LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo](https://mlsanity.com/paper/1809.02966/) (2018)
- [Detecting Vanishing Points using Global Image Context in a Non-Manhattan World](https://mlsanity.com/paper/1608.05684/) (2016)
- [UprightNet: Geometry-Aware Camera Orientation Estimation from Single Images](https://mlsanity.com/paper/1908.07070/) (2019)
- [NeurVPS: Neural Vanishing Point Scanning via Conic Convolution](https://mlsanity.com/paper/1910.06316/) (2019)
- [Taking a Deeper Look at the Inverse Compositional Algorithm](https://mlsanity.com/paper/1812.06861/) (2018)

## Built on by (most cited first, 30 of 52)

- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold](https://mlsanity.com/paper/2505.12549/) (2025)
- [Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation](https://mlsanity.com/paper/2504.14899/) (2025)
- [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025)
- [Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation](https://mlsanity.com/paper/2510.08673/) (2025)
- [EgoSim: Egocentric World Simulator for Embodied Interaction Generation](https://mlsanity.com/paper/2604.01001/) (2026)
- [SpatialTree: How Spatial Abilities Branch Out in MLLMs](https://mlsanity.com/paper/2512.20617/) (2025)
- [Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction](https://mlsanity.com/paper/2602.09013/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [Boost 3D Reconstruction using Diffusion-based Monocular Camera Calibration](https://mlsanity.com/paper/2411.17240/) (2024)
- [UnLoc: Leveraging Depth Uncertainties for Floorplan Localization](https://mlsanity.com/paper/2509.11301/) (2025)
- [EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videos](https://mlsanity.com/paper/2601.01050/) (2026)
- [360Anything: Geometry-Free Lifting of Images and Videos to 360deg](https://mlsanity.com/paper/2601.16192/) (2026)
- [Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D](https://mlsanity.com/paper/2604.05212/) (2026)
- [Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses](https://mlsanity.com/paper/2603.28896/) (2026)
- [Kineo: Calibration-Free Metric Motion Capture From Sparse RGB Cameras](https://mlsanity.com/paper/2510.24464/) (2025)
- [CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation](https://mlsanity.com/paper/2605.12938/) (2026)
- [S-MUSt3R: Sliding Multi-view 3D Reconstruction](https://mlsanity.com/paper/2602.04517/) (2026)
- [Sekai2: From World Exploration to Interactive World Modeling](https://mlsanity.com/paper/2608.09449/) (2026)
- [CAM3R: Camera-Agnostic Model for 3D Reconstruction](https://mlsanity.com/paper/2603.22631/) (2026)
- [CalibAnyView: Beyond Single-View Camera Calibration in the Wild](https://mlsanity.com/paper/2605.14615/) (2026)
- [VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion](https://mlsanity.com/paper/2607.27194/) (2026)
- [MINT: A Unified Model for World-Space Camera and Hand Motion Estimation from Scalable Egocentric Pipeline Supervision](https://mlsanity.com/paper/2609.04958/) (2026)
- [KM-ViPE: Online Tightly Coupled Vision-Language-Geometry Fusion for Open-Vocabulary Semantic SLAM](https://mlsanity.com/paper/2512.01889/) (2025)
- [Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model](https://mlsanity.com/paper/2604.09480/) (2026)
- [SceneAligner: 3D-Grounded Floorplan Localization in the Wild](https://mlsanity.com/paper/2605.22581/) (2026)
- [The Camera Inside the Editor: Reading the Implicit Camera of Image Editors with Painted Calibration Patterns](https://mlsanity.com/paper/2609.37732/) (2026)
- [A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras](https://mlsanity.com/paper/2504.06965/) (2025)
- [Large-scale visual SLAM for in-the-wild videos](https://mlsanity.com/paper/2504.20496/) (2025)

## Abstract (arXiv)

From a single image, visual cues can help deduce intrinsic and extrinsic camera parameters like the focal length and the gravity direction. This single-image calibration can benefit various downstream applications like image editing and 3D mapping. Current approaches to this problem are based on either classical geometry with lines and vanishing points or on deep neural networks trained end-to-end. The learned approaches are more robust but struggle to generalize to new environments and are less accurate than their classical counterparts. We hypothesize that they lack the constraints that 3D geometry provides. In this work, we introduce GeoCalib, a deep neural network that leverages universal rules of 3D geometry through an optimization process. GeoCalib is trained end-to-end to estimate camera parameters and learns to find useful visual cues from the data. Experiments on various benchmarks show that GeoCalib is more robust and more accurate than existing classical and learned approaches. Its internal optimization estimates uncertainties, which help flag failure cases and benefit downstream applications like visual localization. The code and trained models are publicly available at https://github.com/cvg/GeoCalib.