# GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depth

arXiv: https://arxiv.org/abs/2604.11585 · HTML: https://mlsanity.com/paper/2604.11585/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 73 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 30 of 37)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach](https://mlsanity.com/paper/1812.05040/) (2018)
- [Deep Multimodal Fusion by Channel Exchanging](https://mlsanity.com/paper/2011.05005/) (2020)
- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [DADA: Depth-aware Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1904.01886/) (2019)
- [Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2404.04256/) (2024)
- [RGB-D Local Implicit Function for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2104.00622/) (2021)
- [DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation](https://mlsanity.com/paper/2309.09668/) (2023)
- [GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer](https://mlsanity.com/paper/2406.01210/) (2024)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments](https://mlsanity.com/paper/2207.04526/) (2022)
- [DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentation](https://mlsanity.com/paper/2504.04701/) (2025)
- [Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2108.12545/) (2021)
- [Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2304.10756/) (2023)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Unsupervised Semantic Segmentation Through Depth-Guided Feature Correlation and Sampling](https://mlsanity.com/paper/2309.12378/) (2023)
- [Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustness](https://mlsanity.com/paper/2503.18445/) (2025)
- [AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation](https://mlsanity.com/paper/2309.14065/) (2023)
- [Hard Pixel Mining for Depth Privileged Semantic Segmentation](https://mlsanity.com/paper/1906.11437/) (2019)
- [Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.06685/) (2021)
- [Semantic Mapping in Indoor Embodied AI -- A Survey on Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2501.05750/) (2025)

## Built on by (most cited first, 1 of 1)

- [DINOcular: Self-Supervised Visuospatial Representations](https://mlsanity.com/paper/2608.27226/) (2026)

## Abstract (arXiv)

Multimodal perception systems for robotics and embodied AI often assume reliable RGB-D sensing, but in practice, depth is frequently missing, noisy, or corrupted. We thus present GeomPrompt, a lightweight cross-modal adaptation module that synthesizes a task-driven geometric prompt from RGB alone for the fourth channel of a frozen RGB-D semantic segmentation model, without depth supervision. We further introduce GeomPrompt-Recovery, an adaptation module that compensates for degraded depth by predicting the fourth channel correction relevant for the frozen segmenter. Both modules are trained solely with downstream segmentation supervision, enabling recovery of the geometric prior useful for segmentation, rather than estimating depth signals. On SUN RGB-D, GeomPrompt improves over RGB-only inference by +6.1 mIoU on DFormer and +3.0 mIoU on GeminiFusion, while remaining competitive with strong monocular depth estimators. For degraded depth, GeomPrompt-Recovery consistently improves robustness, yielding gains up to +3.6 mIoU under severe depth corruptions. GeomPrompt is also substantially more efficient than monocular depth baselines, reaching 7.8 ms latency versus 38.3 ms and 71.9 ms. These results suggest that task-driven geometric prompting is an efficient mechanism for cross-modal compensation under missing and degraded depth inputs in RGB-D perception.