# Y-Net: A Spatiospectral Dual-Encoder Networkfor Medical Image Segmentation

arXiv: https://arxiv.org/abs/2204.07613 · HTML: https://mlsanity.com/paper/2204.07613/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Focal Frequency Loss for Image Reconstruction and Synthesis](https://mlsanity.com/paper/2012.12821/) (2020)
- [Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation](https://mlsanity.com/paper/1805.02798/) (2018)
- [ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network](https://mlsanity.com/paper/1704.02161/) (2017)
- [Deep Learning based Retinal OCT Segmentation](https://mlsanity.com/paper/1801.09749/) (2018)
- [Multi-scale GCN-assisted two-stage network for joint segmentation of retinal layers and disc in peripapillary OCT images](https://mlsanity.com/paper/2102.04799/) (2021)
- [Y-Net: A deep Convolutional Neural Network for Polyp Detection](https://mlsanity.com/paper/1806.01907/) (2018)
- [U2-Net: A Bayesian U-Net model with epistemic uncertainty feedback for photoreceptor layer segmentation in pathological OCT scans](https://mlsanity.com/paper/1901.07929/) (2019)
- [Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks](https://mlsanity.com/paper/1803.05120/) (2018)

## Built on by (most cited first, 19 of 19)

- [S2ME: Spatial-Spectral Mutual Teaching and Ensemble Learning for Scribble-supervised Polyp Segmentation](https://mlsanity.com/paper/2306.00451/) (2023)
- [SynergyNet: Bridging the Gap between Discrete and Continuous Representations for Precise Medical Image Segmentation](https://mlsanity.com/paper/2310.17764/) (2023)
- [DopUS-Net: Quality-Aware Robotic Ultrasound Imaging based on Doppler Signal](https://mlsanity.com/paper/2305.08938/) (2023)
- [EdgeAL: An Edge Estimation Based Active Learning Approach for OCT Segmentation](https://mlsanity.com/paper/2307.10745/) (2023)
- [Guided Frequency Loss for Image Restoration](https://mlsanity.com/paper/2309.15563/) (2023)
- [Catheter Detection and Segmentation in X-ray Images via Multi-task Learning](https://mlsanity.com/paper/2503.02717/) (2025)
- [Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data](https://mlsanity.com/paper/2207.03448/) (2022)
- [Deep Fourier-embedded Network for RGB and Thermal Salient Object Detection](https://mlsanity.com/paper/2411.18409/) (2024)
- [DualTrack: Sensorless 3D Ultrasound needs Local and Global Context](https://mlsanity.com/paper/2509.09530/) (2025)
- [Shape-Aware Masking for Inpainting in Medical Imaging](https://mlsanity.com/paper/2207.05787/) (2022)
- [DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation](https://mlsanity.com/paper/2304.14571/) (2023)
- [More complex encoder is not all you need](https://mlsanity.com/paper/2309.11139/) (2023)
- [Modular Deep Active Learning Framework for Image Annotation: A Technical Report for the Ophthalmo-AI Project](https://mlsanity.com/paper/2403.15143/) (2024)
- [When Mamba Meets xLSTM: An Efficient and Precise Method with the xLSTM-VMUNet Model for Skin lesion Segmentation](https://mlsanity.com/paper/2411.09363/) (2024)
- [Conformable Convolution for Topologically Aware Learning of Complex Anatomical Structures](https://mlsanity.com/paper/2412.20608/) (2024)
- [3D Reconstruction of Coronary Vessel Trees from Biplanar X-Ray Images Using a Geometric Approach](https://mlsanity.com/paper/2509.13358/) (2025)
- [DeepAf: One-Shot Spatiospectral Auto-Focus Model for Digital Pathology](https://mlsanity.com/paper/2510.05315/) (2025)
- [Phi-SegNet: Phase-Integrated Supervision for Medical Image Segmentation](https://mlsanity.com/paper/2601.16064/) (2026)
- [PC-Seg: Progressive Cross-View Consistency for 3D OCT Segmentation from Sparse 2D Annotations](https://mlsanity.com/paper/2607.17718/) (2026)

## Abstract (arXiv)

Automated segmentation of retinal optical coherence tomography (OCT) images has become an important recent direction in machine learning for medical applications. We hypothesize that the anatomic structure of layers and their high-frequency variation in OCT images make retinal OCT a fitting choice for extracting spectral-domain features and combining them with spatial domain features. In this work, we present Upsilon-Net, an architecture that combines the frequency domain features with the image domain to improve the segmentation performance of OCT images. The results of this work demonstrate that the introduction of two branches, one for spectral and one for spatial domain features, brings a very significant improvement in fluid segmentation performance and allows outperformance as compared to the well-known U-Net model. Our improvement was 13% on the fluid segmentation dice score and 1.9% on the average dice score. Finally, removing selected frequency ranges in the spectral domain demonstrates the impact of these features on the fluid segmentation outperformance.