mlsanityData to 7 Oct 2026

SegRGB-X: General RGB-X Semantic Segmentation Model

arXiv 2603.28023 ↗First posted 30 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Semantic segmentation across arbitrary sensor modalities faces significant challenges due to diverse sensor characteristics, and the traditional configurations for this task result in redundant development efforts. We address these challenges by introducing a universal arbitrary-modal semantic segmentation framework that unifies segmentation across multiple modalities. Our approach features three key innovations: (1) the Modality-aware CLIP (MA-CLIP), which provides modality-specific scene understanding guidance through LoRA fine-tuning; (2) Modality-aligned Embeddings for capturing fine-grained features; and (3) the Domain-specific Refinement Module (DSRM) for dynamic feature adjustment. Evaluated on five diverse datasets with different complementary modalities (event, thermal, depth, polarization, and light field), our model surpasses specialized multi-modal methods and achieves state-of-the-art performance with a mIoU of 65.03%. The codes will be released upon acceptance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 40 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

24 of 40 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  8. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  9. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 229
  10. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 115
  11. 2023Controlling Vision-Language Models for Multi-Task Image Restorationcited by 108
  12. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 105
12 more
  1. 2021ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic Segmentationcited by 75
  2. 2018Depth-aware CNN for RGB-D Segmentationcited by 73
  3. 2024GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformercited by 42
  4. 2024StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentationcited by 38
  5. 2024Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalitiescited by 29
  6. 2024Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detectioncited by 27
  7. 2020Malleable 2.5D Convolution: Learning Receptive Fields along the Depth-axis for RGB-D Scene Parsingcited by 24
  8. 2020Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusioncited by 23
  9. 2021Exploring Event-driven Dynamic Context for Accident Scene Segmentationcited by 22
  10. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 21
  11. 2020Depth-Adapted CNN for RGB-D camerascited by 12
  12. 2024MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selectioncited by 12

Built on by

Not cited yet.