# FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration

arXiv: https://arxiv.org/abs/2207.01925 · HTML: https://mlsanity.com/paper/2207.01925/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 48 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Blind Geometric Distortion Correction on Images Through Deep Learning](https://mlsanity.com/paper/1909.03459/) (2019)
- [Learning to Calibrate Straight Lines for Fisheye Image Rectification](https://mlsanity.com/paper/1904.09856/) (2019)
- [Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow](https://mlsanity.com/paper/2103.16026/) (2021)
- [Robust Navigation of a Soft Growing Robot by Exploiting Contact with the Environment](https://mlsanity.com/paper/1908.08645/) (2019)

## Built on by (most cited first, 9 of 9)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers](https://mlsanity.com/paper/2306.12992/) (2023)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [Exploiting the Distortion-Semantic Interaction in Fisheye Data](https://mlsanity.com/paper/2305.00079/) (2023)
- [Sector Patch Embedding: An Embedding Module Conforming to The Distortion Pattern of Fisheye Image](https://mlsanity.com/paper/2303.14645/) (2023)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)
- [QueryCDR: Query-Based Controllable Distortion Rectification Network for Fisheye Images](https://mlsanity.com/paper/2412.13496/) (2024)
- [FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters](https://mlsanity.com/paper/2610.02799/) (2026)

## Abstract (arXiv)

Numerous significant progress on fisheye image rectification has been achieved through CNN. Nevertheless, constrained by a fixed receptive field, the global distribution and the local symmetry of the distortion have not been fully exploited. To leverage these two characteristics, we introduced Fishformer that processes the fisheye image as a sequence to enhance global and local perception. We tuned the Transformer according to the structural properties of fisheye images. First, the uneven distortion distribution in patches generated by the existing square slicing method confuses the network, resulting in difficult training. Therefore, we propose an annulus slicing method to maintain the consistency of the distortion in each patch, thus perceiving the distortion distribution well. Second, we analyze that different distortion parameters have their own efficacy domains. Hence, the perception of the local area is as important as the global, but Transformer has a weakness for local texture perception. Therefore, we propose a novel layer attention mechanism to enhance the local perception and texture transfer. Our network simultaneously implements global perception and focused local perception decided by the different parameters. Extensive experiments demonstrate that our method provides superior performance compared with state-of-the-art methods.