# Convolution kernel adaptation to calibrated fisheye

arXiv: https://arxiv.org/abs/2402.01456 · HTML: https://mlsanity.com/paper/2402.01456/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimation](https://mlsanity.com/paper/2102.03550/) (2021)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth](https://mlsanity.com/paper/1904.02028/) (2019)
- [OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion](https://mlsanity.com/paper/2203.00838/) (2022)
- [Corners for Layout: End-to-End Layout Recovery from 360 Images](https://mlsanity.com/paper/1903.08094/) (2019)
- [Spherical View Synthesis for Self-Supervised 360 Depth Estimation](https://mlsanity.com/paper/1909.08112/) (2019)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [ACDNet: Adaptively Combined Dilated Convolution for Monocular Panorama Depth Estimation](https://mlsanity.com/paper/2112.14440/) (2021)
- [Active Convolution: Learning the Shape of Convolution for Image Classification](https://mlsanity.com/paper/1703.09076/) (2017)
- [Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision](https://mlsanity.com/paper/1803.06192/) (2018)
- [What's in my Room? Object Recognition on Indoor Panoramic Images](https://mlsanity.com/paper/1910.06138/) (2019)
- [End to End Vehicle Lateral Control Using a Single Fisheye Camera](https://mlsanity.com/paper/1808.06940/) (2018)
- [FreDSNet: Joint Monocular Depth and Semantic Segmentation with Fast Fourier Convolutions](https://mlsanity.com/paper/2210.01595/) (2022)
- [Real-time Detection, Tracking, and Classification of Moving and Stationary Objects using Multiple Fisheye Images](https://mlsanity.com/paper/1803.06077/) (2018)

## Built on by (most cited first, 5 of 5)

- [AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration](https://mlsanity.com/paper/2503.12701/) (2025)
- [Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras](https://mlsanity.com/paper/2606.08680/) (2026)
- [Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups](https://mlsanity.com/paper/2606.08844/) (2026)
- [From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation](https://mlsanity.com/paper/2608.27860/) (2026)
- [FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters](https://mlsanity.com/paper/2610.02799/) (2026)

## Abstract (arXiv)

Convolution kernels are the basic structural component of convolutional neural networks (CNNs). In the last years there has been a growing interest in fisheye cameras for many applications. However, the radially symmetric projection model of these cameras produces high distortions that affect the performance of CNNs, especially when the field of view is very large. In this work, we tackle this problem by proposing a method that leverages the calibration of cameras to deform the convolution kernel accordingly and adapt to the distortion. That way, the receptive field of the convolution is similar to standard convolutions in perspective images, allowing us to take advantage of pre-trained networks in large perspective datasets. We show how, with just a brief fine-tuning stage in a small dataset, we improve the performance of the network for the calibrated fisheye with respect to standard convolutions in depth estimation and semantic segmentation.