# FreDSNet: Joint Monocular Depth and Semantic Segmentation with Fast Fourier Convolutions

arXiv: https://arxiv.org/abs/2210.01595 · HTML: https://mlsanity.com/paper/2210.01595/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth](https://mlsanity.com/paper/1904.02028/) (2019)
- [Corners for Layout: End-to-End Layout Recovery from 360 Images](https://mlsanity.com/paper/1903.08094/) (2019)
- [BlitzNet: A Real-Time Deep Network for Scene Understanding](https://mlsanity.com/paper/1708.02813/) (2017)
- [Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey](https://mlsanity.com/paper/1803.03352/) (2018)
- [Striking the Right Balance: Recall Loss for Semantic Segmentation](https://mlsanity.com/paper/2106.14917/) (2021)
- [What's in my Room? Object Recognition on Indoor Panoramic Images](https://mlsanity.com/paper/1910.06138/) (2019)

## Built on by (most cited first, 11 of 11)

- [SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation](https://mlsanity.com/paper/2306.03403/) (2023)
- [Open Panoramic Segmentation](https://mlsanity.com/paper/2407.02685/) (2024)
- [Convolution kernel adaptation to calibrated fisheye](https://mlsanity.com/paper/2402.01456/) (2024)
- [The Power of Linear Combinations: Learning with Random Convolutions](https://mlsanity.com/paper/2301.11360/) (2023)
- [Volumetric Fast Fourier Convolution for Detecting Ink on the Carbonized Herculaneum Papyri](https://mlsanity.com/paper/2308.05070/) (2023)
- [FastViDAR: Real-Time Omnidirectional Depth Estimation via Alternative Hierarchical Attention](https://mlsanity.com/paper/2509.23733/) (2025)
- [Local Contrast and Global Contextual Information Make Infrared Small Object Salient Again](https://mlsanity.com/paper/2301.12093/) (2023)
- [Calibrating Panoramic Depth Estimation for Practical Localization and Mapping](https://mlsanity.com/paper/2308.14005/) (2023)
- [Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation](https://mlsanity.com/paper/2603.25131/) (2026)
- [SFA-UNet: More Attention to Multi-Scale Contrast and Contextual Information in Infrared Small Object Segmentation](https://mlsanity.com/paper/2410.22881/) (2024)
- [PAGCNet: A Pose-Aware and Geometry Constrained Framework for Panoramic Depth Estimation](https://mlsanity.com/paper/2510.07817/) (2025)

## Abstract (arXiv)

In this work we present FreDSNet, a deep learning solution which obtains semantic 3D understanding of indoor environments from single panoramas. Omnidirectional images reveal task-specific advantages when addressing scene understanding problems due to the 360-degree contextual information about the entire environment they provide. However, the inherent characteristics of the omnidirectional images add additional problems to obtain an accurate detection and segmentation of objects or a good depth estimation. To overcome these problems, we exploit convolutions in the frequential domain obtaining a wider receptive field in each convolutional layer. These convolutions allow to leverage the whole context information from omnidirectional images. FreDSNet is the first network that jointly provides monocular depth estimation and semantic segmentation from a single panoramic image exploiting fast Fourier convolutions. Our experiments show that FreDSNet has similar performance as specific state of the art methods for semantic segmentation and depth estimation. FreDSNet code is publicly available in https://github.com/Sbrunoberenguel/FreDSNet