# Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow

arXiv: https://arxiv.org/abs/2103.16026 · HTML: https://mlsanity.com/paper/2103.16026/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 48 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 10 of 10)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Target-Aware Deep Tracking](https://mlsanity.com/paper/1904.01772/) (2019)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Blind Geometric Distortion Correction on Images Through Deep Learning](https://mlsanity.com/paper/1909.03459/) (2019)
- [Learning to Calibrate Straight Lines for Fisheye Image Rectification](https://mlsanity.com/paper/1904.09856/) (2019)

## Built on by (most cited first, 23 of 23)

- [DocScanner: Robust Document Image Rectification with Progressive Learning](https://mlsanity.com/paper/2110.14968/) (2021)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Self-Supervised Camera Self-Calibration from Video](https://mlsanity.com/paper/2112.03325/) (2021)
- [Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers](https://mlsanity.com/paper/2306.12992/) (2023)
- [SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning](https://mlsanity.com/paper/2308.09040/) (2023)
- [Curved Diffusion: A Generative Model With Optical Geometry Control](https://mlsanity.com/paper/2311.17609/) (2023)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration](https://mlsanity.com/paper/2207.01925/) (2022)
- [RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning](https://mlsanity.com/paper/2301.01661/) (2023)
- [MOWA: Multiple-in-One Image Warping Model](https://mlsanity.com/paper/2404.10716/) (2024)
- [Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion](https://mlsanity.com/paper/2111.12927/) (2021)
- [Dual Diffusion Architecture for Fisheye Image Rectification: Synthetic-to-Real Generalization](https://mlsanity.com/paper/2301.11785/) (2023)
- [A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods](https://mlsanity.com/paper/2401.00442/) (2023)
- [The Multi-Modal Video Reasoning and Analyzing Competition](https://mlsanity.com/paper/2108.08344/) (2021)
- [Spatiotemporal Deformation Perception for Fisheye Video Rectification](https://mlsanity.com/paper/2302.03934/) (2023)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)
- [QueryCDR: Query-Based Controllable Distortion Rectification Network for Fisheye Images](https://mlsanity.com/paper/2412.13496/) (2024)
- [Neural Camera Models](https://mlsanity.com/paper/2208.12903/) (2022)
- [Location-guided Head Pose Estimation for Fisheye Image](https://mlsanity.com/paper/2402.18320/) (2024)
- [RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation](https://mlsanity.com/paper/2406.18927/) (2024)
- [A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras](https://mlsanity.com/paper/2504.06965/) (2025)
- [Rectification Reimagined: A Unified Mamba Model for Image Correction and Rectangling with Prompts](https://mlsanity.com/paper/2512.18718/) (2025)
- [Physics-aware Masked Diffusion-based Flood Simulation for Urban Fisheye Disaster Detection](https://mlsanity.com/paper/2607.15527/) (2026)

## Abstract (arXiv)

Distortion rectification is often required for fisheye images. The generation-based method is one mainstream solution due to its label-free property, but its naive skip-connection and overburdened decoder will cause blur and incomplete correction. First, the skip-connection directly transfers the image features, which may introduce distortion and cause incomplete correction. Second, the decoder is overburdened during simultaneously reconstructing the content and structure of the image, resulting in vague performance. To solve these two problems, in this paper, we focus on the interpretable correction mechanism of the distortion rectification network and propose a feature-level correction scheme. We embed a correction layer in skip-connection and leverage the appearance flows in different layers to pre-correct the image features. Consequently, the decoder can easily reconstruct a plausible result with the remaining distortion-less information. In addition, we propose a parallel complementary structure. It effectively reduces the burden of the decoder by separating content reconstruction and structure correction. Subjective and objective experiment results on different datasets demonstrate the superiority of our method.