# FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens

arXiv: https://arxiv.org/abs/2206.05844 · HTML: https://mlsanity.com/paper/2206.05844/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 12 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020)
- [Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting](https://mlsanity.com/paper/1904.07475/) (2019)
- [High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis](https://mlsanity.com/paper/1611.09969/) (2016)
- [Boundless: Generative Adversarial Networks for Image Extension](https://mlsanity.com/paper/1908.07007/) (2019)
- [Very Long Natural Scenery Image Prediction by Outpainting](https://mlsanity.com/paper/1912.12688/) (2019)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Painting Outside the Box: Image Outpainting with GANs](https://mlsanity.com/paper/1808.08483/) (2018)
- [Blind Geometric Distortion Correction on Images Through Deep Learning](https://mlsanity.com/paper/1909.03459/) (2019)

## Built on by (most cited first, 2 of 2)

- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformer](https://mlsanity.com/paper/2211.11293/) (2022)

## Abstract (arXiv)

Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV). However, the fisheye image generally contains invalid black regions induced by its imaging model. In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions, improving the integrity of captured scenes. Compared with the rectangle and undistorted image, there are two challenges for fisheye image outpainting: irregular painting regions and distortion synthesis. Observing the radial symmetry of the fisheye image, we first propose a polar outpainting strategy to extrapolate the coherent semantics from the center to the outside region. Such an outpainting manner considers the distribution pattern of radial distortion and the circle boundary, boosting a more reasonable completion direction. For the distortion synthesis, we propose a spiral distortion-aware perception module, in which the learning path keeps consistent with the distortion prior of the fisheye image. Subsequently, a scene revision module rearranges the generated pixels with the estimated distortion to match the fisheye image, thus extending the FoV. In the experiment, we evaluate the proposed FisheyeEX on three popular outdoor datasets: Cityscapes, BDD100k, and KITTI, and one real-world fisheye image dataset. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.