# FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition

arXiv: https://arxiv.org/abs/2203.07255 · HTML: https://mlsanity.com/paper/2203.07255/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 10 of 10)

- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Kernel Transformer Networks for Compact Spherical Convolution](https://mlsanity.com/paper/1812.03115/) (2018)
- [Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras](https://mlsanity.com/paper/1801.00708/) (2018)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Universal Semantic Segmentation for Fisheye Urban Driving Images](https://mlsanity.com/paper/2002.03736/) (2020)
- [Active Convolution: Learning the Shape of Convolution for Image Classification](https://mlsanity.com/paper/1703.09076/) (2017)
- [Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems](https://mlsanity.com/paper/2102.10191/) (2021)

## Built on by (most cited first, 10 of 10)

- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [Hyperbolic Random Forests](https://mlsanity.com/paper/2308.13279/) (2023)
- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [Exploiting the Distortion-Semantic Interaction in Fisheye Data](https://mlsanity.com/paper/2305.00079/) (2023)
- [Hyperbolic Geometry in Computer Vision: A Survey](https://mlsanity.com/paper/2304.10764/) (2023)
- [Hyperbolic Convolution via Kernel Point Aggregation](https://mlsanity.com/paper/2306.08862/) (2023)
- [Why are hyperbolic neural networks effective? A study on hierarchical representation capability](https://mlsanity.com/paper/2402.02478/) (2024)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)

## Abstract (arXiv)

Conventional convolution neural networks (CNNs) trained on narrow Field-of-View (FoV) images are the state-of-the-art approaches for object recognition tasks. Some methods proposed the adaptation of CNNs to ultra-wide FoV images by learning deformable kernels. However, they are limited by the Euclidean geometry and their accuracy degrades under strong distortions caused by fisheye projections. In this work, we demonstrate that learning the shape of convolution kernels in non-Euclidean spaces is better than existing deformable kernel methods. In particular, we propose a new approach that learns deformable kernel parameters (positions) in hyperbolic space. FisheyeHDK is a hybrid CNN architecture combining hyperbolic and Euclidean convolution layers for positions and features learning. First, we provide an intuition of hyperbolic space for wide FoV images. Using synthetic distortion profiles, we demonstrate the effectiveness of our approach. We select two datasets - Cityscapes and BDD100K 2020 - of perspective images which we transform to fisheye equivalents at different scaling factors (analog to focal lengths). Finally, we provide an experiment on data collected by a real fisheye camera. Validations and experiments show that our approach improves existing deformable kernel methods for CNN adaptation on fisheye images.