mlsanityData to 7 Oct 2026

AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation

arXiv 2309.14065 ↗First posted 25 Sep 2023cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Understanding indoor scenes is crucial for urban studies. Considering the dynamic nature of indoor environments, effective semantic segmentation requires both real-time operation and high accuracy.To address this, we propose AsymFormer, a novel network that improves real-time semantic segmentation accuracy using RGB-D multi-modal information without substantially increasing network complexity. AsymFormer uses an asymmetrical backbone for multimodal feature extraction, reducing redundant parameters by optimizing computational resource distribution. To fuse asymmetric multimodal features, a Local Attention-Guided Feature Selection (LAFS) module is used to selectively fuse features from different modalities by leveraging their dependencies. Subsequently, a Cross-Modal Attention-Guided Feature Correlation Embedding (CMA) module is introduced to further extract cross-modal representations. The AsymFormer demonstrates competitive results with 54.1% mIoU on NYUv2 and 49.1% mIoU on SUNRGBD. Notably, AsymFormer achieves an inference speed of 65 FPS (79 FPS after implementing mixed precision quantization) on RTX3090, demonstrating that AsymFormer can strike a balance between high accuracy and efficiency.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

34 of 51 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  9. 2016Pyramid Scene Parsing Networkcited by 2,472
  10. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  11. 2017Non-local Neural Networkscited by 2,290
  12. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
22 more
  1. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  2. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  3. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  4. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  5. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  6. 2021Coordinate Attention for Efficient Mobile Network Designcited by 239
  7. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 229
  8. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  9. 2019ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blockscited by 178
  10. 2022Multimodal Token Fusion for Vision Transformerscited by 132
  11. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 120
  12. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 105
  13. 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 83
  14. 2020Temporally Distributed Networks for Fast Video Semantic Segmentationcited by 77
  15. 2021ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic Segmentationcited by 75
  16. 2018RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentationcited by 65
  17. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  18. 2023DFormer: Rethinking RGBD Representation Learning for Semantic Segmentationcited by 58
  19. 2018Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotationscited by 51
  20. 2016Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networkscited by 35
  21. 2022Efficient Multi-Task RGB-D Scene Analysis for Indoor Environmentscited by 32
  22. 2018Light-Weight RefineNet for Real-Time Semantic Segmentationcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 6.

16 papers.

Sort:

By area:

  1. 2025DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentationcited by 30
  2. 2024HAPNet: Toward Superior RGB-Thermal Scene Parsing via Hybrid, Asymmetric, and Progressive Heterogeneous Feature Fusioncited by 10
  3. 2025Stronger, Steadier & Superior: Geometric Consistency in Depth VFM Forges Domain Generalized Semantic Segmentationcited by 6
  4. 2024RT-GS2: Real-Time Generalizable Semantic Segmentation for 3D Gaussian Representations of Radiance Fieldscited by 5
  5. 2025Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Imagescited by 3
  6. 2025LuSeg: Efficient Negative and Positive Obstacles Segmentation via Contrast-Driven Multi-Modal Feature Fusion on the Lunarcited by 2
  7. 2025Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completioncited by 2
  8. 2025DiffPixelFormer: Differential Pixel-Aware Transformer for RGB-D Indoor Scene Segmentationcited by 2
  9. 2024Self-supervised cost of transport estimation for multimodal path planningcited by 1
  10. 2025HDBFormer: Efficient RGB-D Semantic Segmentation with A Heterogeneous Dual-Branch Frameworkcited by 1
  11. 2026GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depthcited by 1
  12. 2026RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodingscited by 1
4 more
  1. 2026NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulationcited by 0
  2. 2026Toward Reliable RGB-D Semantic Segmentation: Handling Missing Modalities via Condition Dropoutcited by 0
  3. 2026URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentationcited by 0
  4. 2026A Critical Synthesis of Uncertainty Quantification and Foundation Models for Semantic Segmentationcited by 0