# CTRL-C: Camera calibration TRansformer with Line-Classification

arXiv: https://arxiv.org/abs/2109.02259 · HTML: https://mlsanity.com/paper/2109.02259/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 6 Sep 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [12-in-1: Multi-Task Vision and Language Representation Learning](https://mlsanity.com/paper/1912.02315/) (2019)
- [Line Segment Detection Using Transformers without Edges](https://mlsanity.com/paper/2101.01909/) (2021)
- [Rendering Synthetic Objects into Legacy Photographs](https://mlsanity.com/paper/1912.11565/) (2019)
- [A Perceptual Measure for Deep Single Image Camera Calibration](https://mlsanity.com/paper/1712.01259/) (2017)
- [Automatic Scene Inference for 3D Object Compositing](https://mlsanity.com/paper/1912.12297/) (2019)
- [Horizon Lines in the Wild](https://mlsanity.com/paper/1604.02129/) (2016)
- [HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures](https://mlsanity.com/paper/2008.03286/) (2020)
- [Detecting Vanishing Points using Global Image Context in a Non-Manhattan World](https://mlsanity.com/paper/1608.05684/) (2016)
- [UprightNet: Geometry-Aware Camera Orientation Estimation from Single Images](https://mlsanity.com/paper/1908.07070/) (2019)
- [Single View Metrology in the Wild](https://mlsanity.com/paper/2007.09529/) (2020)
- [Neural Geometric Parser for Single Image Camera Calibration](https://mlsanity.com/paper/2007.11855/) (2020)

## Built on by (most cited first, 30 of 33)

- [MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision](https://mlsanity.com/paper/2410.19115/) (2024)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [Perspective Fields for Single Image Camera Calibration](https://mlsanity.com/paper/2212.03239/) (2022)
- [CameraHMR: Aligning People with Perspective](https://mlsanity.com/paper/2411.08128/) (2024)
- [Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View](https://mlsanity.com/paper/2404.03421/) (2024)
- [Future Transformer for Long-term Action Anticipation](https://mlsanity.com/paper/2205.14022/) (2022)
- [GeoCalib: Learning Single-image Calibration with Geometric Optimization](https://mlsanity.com/paper/2409.06704/) (2024)
- [Tame a Wild Camera: In-the-Wild Monocular Camera Calibration](https://mlsanity.com/paper/2306.10988/) (2023)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Collaborative Transformers for Grounded Situation Recognition](https://mlsanity.com/paper/2203.16518/) (2022)
- [Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation](https://mlsanity.com/paper/2510.08673/) (2025)
- [AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration](https://mlsanity.com/paper/2503.12701/) (2025)
- [DiffCalib: Reformulating Monocular Camera Calibration as Diffusion-Based Dense Incident Map Generation](https://mlsanity.com/paper/2405.15619/) (2024)
- [Deep Rotation Correction without Angle Prior](https://mlsanity.com/paper/2207.03054/) (2022)
- [Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion](https://mlsanity.com/paper/2111.12927/) (2021)
- [A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation](https://mlsanity.com/paper/2207.12934/) (2022)
- [RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation](https://mlsanity.com/paper/2508.13968/) (2025)
- [SensorX2car: Sensors-to-car calibration for autonomous driving in road scenarios](https://mlsanity.com/paper/2301.07279/) (2023)
- [Deep Single Image Camera Calibration by Heatmap Regression to Recover Fisheye Images Under Manhattan World Assumption](https://mlsanity.com/paper/2303.17166/) (2023)
- [Floating No More: Object-Ground Reconstruction from a Single Image](https://mlsanity.com/paper/2407.18914/) (2024)
- [SOFI: Multi-Scale Deformable Transformer for Camera Calibration with Enhanced Line Queries](https://mlsanity.com/paper/2409.15553/) (2024)
- [AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion](https://mlsanity.com/paper/2503.21581/) (2025)
- [CalibAnyView: Beyond Single-View Camera Calibration in the Wild](https://mlsanity.com/paper/2605.14615/) (2026)
- [Multi-HMR 2: Multi-Person Camera-Centric Human Detection, Mesh Recovery and Tracking](https://mlsanity.com/paper/2606.14841/) (2026)
- [fCOP: Focal Length Estimation from Category-level Object Priors](https://mlsanity.com/paper/2409.19641/) (2024)
- [StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision](https://mlsanity.com/paper/2603.29368/) (2026)
- [U-ARE-ME: Uncertainty-Aware Rotation Estimation in Manhattan Environments](https://mlsanity.com/paper/2403.15583/) (2024)
- [CoL3D: Collaborative Learning of Single-view Depth and Camera Intrinsics for Metric 3D Shape Recovery](https://mlsanity.com/paper/2502.08902/) (2025)
- [Tensor-Based Self-Calibration of Cameras via the TrifocalCalib Method](https://mlsanity.com/paper/2509.17620/) (2025)
- [Otter: Mitigating Background Distractions of Wide-Angle Few-Shot Action Recognition with Enhanced RWKV](https://mlsanity.com/paper/2511.06741/) (2025)

## Abstract (arXiv)

Single image camera calibration is the task of estimating the camera parameters from a single input image, such as the vanishing points, focal length, and horizon line. In this work, we propose Camera calibration TRansformer with Line-Classification (CTRL-C), an end-to-end neural network-based approach to single image camera calibration, which directly estimates the camera parameters from an image and a set of line segments. Our network adopts the transformer architecture to capture the global structure of an image with multi-modal inputs in an end-to-end manner. We also propose an auxiliary task of line classification to train the network to extract the global geometric information from lines effectively. Our experiments demonstrate that CTRL-C outperforms the previous state-of-the-art methods on the Google Street View and SUN360 benchmark datasets.