# Exploiting the Distortion-Semantic Interaction in Fisheye Data

arXiv: https://arxiv.org/abs/2305.00079 · HTML: https://mlsanity.com/paper/2305.00079/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 28 Apr 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Backpropagated Gradient Representations for Anomaly Detection](https://mlsanity.com/paper/2007.09507/) (2020)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models](https://mlsanity.com/paper/2007.06676/) (2020)
- [CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition](https://mlsanity.com/paper/1712.02463/) (2017)
- [FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification](https://mlsanity.com/paper/1804.04784/) (2018)
- [Learning to Calibrate Straight Lines for Fisheye Image Rectification](https://mlsanity.com/paper/1904.09856/) (2019)
- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems](https://mlsanity.com/paper/2102.10191/) (2021)
- [3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image](https://mlsanity.com/paper/2003.03759/) (2020)
- [Volumetric Supervised Contrastive Learning for Seismic Semantic Segmentation](https://mlsanity.com/paper/2206.08158/) (2022)
- [FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition](https://mlsanity.com/paper/2203.07255/) (2022)
- [Computer vision in automated parking systems: Design, implementation and challenges](https://mlsanity.com/paper/2104.12537/) (2021)
- [FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration](https://mlsanity.com/paper/2207.01925/) (2022)
- [Clinical Contrastive Learning for Biomarker Detection](https://mlsanity.com/paper/2211.05092/) (2022)

## Built on by (most cited first, 6 of 6)

- [FOCAL: A Cost-Aware Video Dataset for Active Learning](https://mlsanity.com/paper/2311.10591/) (2023)
- [HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms](https://mlsanity.com/paper/2410.23200/) (2024)
- [Hierarchical and Multimodal Data for Daily Activity Understanding](https://mlsanity.com/paper/2504.17696/) (2025)
- [AdaDim: Dimensionality Adaptation for SSL Representational Dynamics](https://mlsanity.com/paper/2505.12576/) (2025)
- [MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization](https://mlsanity.com/paper/2605.01967/) (2026)
- [SynCo: Learning Cross-Modal Synergy by Contrasting Interaction Residuals](https://mlsanity.com/paper/2609.32846/) (2026)

## Abstract (arXiv)

In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality. Fisheye data has the wider field of view advantage over other types of cameras, but this comes at the expense of high radial distortion. As a result, objects further from the center exhibit deformations that make it difficult for a model to identify their semantic context. While previous work has attempted architectural and training augmentation changes to alleviate this effect, no work has attempted to guide the model towards learning a representation space that reflects this interaction between distortion and semantic context inherent to fisheye data. We introduce an approach to exploit this relationship by first extracting distortion class labels based on an object's distance from the center of the image. We then shape a backbone's representation space with a weighted contrastive loss that constrains objects of the same semantic class and distortion class to be close to each other within a lower dimensional embedding space. This backbone trained with both semantic and distortion information is then fine-tuned within an object detection setting to empirically evaluate the quality of the learnt representation. We show this method leads to performance improvements by as much as 1.1% mean average precision over standard object detection strategies and .6% improvement over other state of the art representation learning approaches.