# SIR: Self-supervised Image Rectification via Seeing the Same Scene from Multiple Different Lenses

arXiv: https://arxiv.org/abs/2011.14611 · HTML: https://mlsanity.com/paper/2011.14611/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning](https://mlsanity.com/paper/2308.09040/) (2023)
- [A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods](https://mlsanity.com/paper/2401.00442/) (2023)

## Abstract (arXiv)

Deep learning has demonstrated its power in image rectification by leveraging the representation capacity of deep neural networks via supervised training based on a large-scale synthetic dataset. However, the model may overfit the synthetic images and generalize not well on real-world fisheye images due to the limited universality of a specific distortion model and the lack of explicitly modeling the distortion and rectification process. In this paper, we propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of a same scene from different lens should be the same. Specifically, we devise a new network architecture with a shared encoder and several prediction heads, each of which predicts the distortion parameter of a specific distortion model. We further leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters and exploit the intra- and inter-model consistency between them during training, thereby leading to a self-supervised learning scheme without the need for ground-truth distortion parameters or normal images. Experiments on synthetic dataset and real-world fisheye images demonstrate that our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods. Self-supervised learning also improves the universality of distortion models while keeping their self-consistency.