# Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

arXiv: https://arxiv.org/abs/2111.12927 · HTML: https://mlsanity.com/paper/2111.12927/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 30 of 32)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation](https://mlsanity.com/paper/1901.02985/) (2019)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [Mish: A Self Regularized Non-Monotonic Activation Function](https://mlsanity.com/paper/1908.08681/) (2019)
- [Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Camera](https://mlsanity.com/paper/1907.05820/) (2019)
- [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017)
- [STMTrack: Template-free Visual Tracking with Space-time Memory Networks](https://mlsanity.com/paper/2104.00324/) (2021)
- [A Perceptual Measure for Deep Single Image Camera Calibration](https://mlsanity.com/paper/1712.01259/) (2017)
- [The StreetLearn Environment and Dataset](https://mlsanity.com/paper/1903.01292/) (2019)
- [Multi-Loss Weighting with Coefficient of Variations](https://mlsanity.com/paper/2009.01717/) (2020)
- [Distilling Knowledge From a Deep Pose Regressor Network](https://mlsanity.com/paper/1908.00858/) (2019)
- [CTRL-C: Camera calibration TRansformer with Line-Classification](https://mlsanity.com/paper/2109.02259/) (2021)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Blind Geometric Distortion Correction on Images Through Deep Learning](https://mlsanity.com/paper/1909.03459/) (2019)
- [UprightNet: Geometry-Aware Camera Orientation Estimation from Single Images](https://mlsanity.com/paper/1908.07070/) (2019)
- [Local Supports Global: Deep Camera Relocalization with Sequence Enhancement](https://mlsanity.com/paper/1908.04391/) (2019)
- [Learning to Calibrate Straight Lines for Fisheye Image Rectification](https://mlsanity.com/paper/1904.09856/) (2019)
- [Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow](https://mlsanity.com/paper/2103.16026/) (2021)
- [A Deep Ordinal Distortion Estimation Approach for Distortion Rectification](https://mlsanity.com/paper/2007.10689/) (2020)
- [Neural Geometric Parser for Single Image Camera Calibration](https://mlsanity.com/paper/2007.11855/) (2020)

## Built on by (most cited first, 6 of 6)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration](https://mlsanity.com/paper/2503.12701/) (2025)
- [RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning](https://mlsanity.com/paper/2301.01661/) (2023)
- [Deep Single Image Camera Calibration by Heatmap Regression to Recover Fisheye Images Under Manhattan World Assumption](https://mlsanity.com/paper/2303.17166/) (2023)
- [A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods](https://mlsanity.com/paper/2401.00442/) (2023)
- [Panoramic Distortion-Aware Tokenization for Person Detection and Localization in Overhead Fisheye Images](https://mlsanity.com/paper/2503.14228/) (2025)

## Abstract (arXiv)

Although recent learning-based calibration methods can predict extrinsic and intrinsic camera parameters from a single image, the accuracy of these methods is degraded in fisheye images. This degradation is caused by mismatching between the actual projection and expected projection. To address this problem, we propose a generic camera model that has the potential to address various types of distortion. Our generic camera model is utilized for learning-based methods through a closed-form numerical calculation of the camera projection. Simultaneously to recover rotation and fisheye distortion, we propose a learning-based calibration method that uses the camera model. Furthermore, we propose a loss function that alleviates the bias of the magnitude of errors for four extrinsic and intrinsic camera parameters. Extensive experiments demonstrated that our proposed method outperformed conventional methods on two largescale datasets and images captured by off-the-shelf fisheye cameras. Moreover, we are the first researchers to analyze the performance of learning-based methods using various types of projection for off-the-shelf cameras.