mlsanityData to 14 Oct 2026

Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classification

arXiv 2606.05455 ↗First posted 3 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The missing-modality problem poses a significant challenge in image-tabular multimodal learning across a wide range of multimedia applications, including product understanding, recommendation systems, and medical diagnosis. This challenge is particularly pronounced when the two modalities are highly heterogeneous, as images and tabular attributes differ substantially in their semantic granularity and data distributions. Existing methods learn modality-invariant representations through disentanglement and alignment over global token-averaged features, capturing only coarse cross-modal consistency and overlooking fine-grained semantic and distributional misalignment, which hampers the exploitation of complementary cues under missing modalities. To address this, we propose DFPL, a novel framework for fine-grained prototype learning. Specifically, Shared-Specific Prototype Modeling (SSPM) extracts compact and diverse shared and modality-specific prototypes, and further performs prototype-level disentanglement to suppress redundant intra-modality correlations. Additionally, we propose a Prototype-guided Fine-grained Alignment (PFA) module that jointly enforces prototype-level distribution matching and prototype-to-class semantic alignment within a unified prototype space, thereby preserving both fine-grained distributional and semantic consistency across modalities. We further introduce a Class-aware Multi-scale Aggregation (CMA) module to adaptively aggregate shared semantics and modality-specific characteristics from global and prototype levels for robust predictions. Extensive experiments on three diverse image-tabular benchmarks demonstrate the superiority of our method compared to the previous approaches under various missing-modality settings. Code will be made publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 55 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

35 of 55 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  6. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  7. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  8. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
  9. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 128
  10. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  11. 2025FG-CLIP: Fine-Grained Visual and Textual Alignmentcited by 81
  12. 2020Faster Wasserstein Distance Estimation with the Sinkhorn Divergencecited by 73
23 more
  1. 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 69
  2. 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understandingcited by 54
  3. 2024DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistencycited by 53
  4. 2025Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detectioncited by 48
  5. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 45
  6. 2023Unified Multi-Modal Image Synthesis for Missing Modality Imputationcited by 40
  7. 2021Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transformcited by 38
  8. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  9. 2024TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Datacited by 32
  10. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 24
  11. 2021Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillationcited by 22
  12. 2024Deep Correlated Prompting for Visual Recognition with Missing Modalitiescited by 18
  13. 2021DVM-CAR: A large-scale automotive dataset for visual marketing research and applicationscited by 17
  14. 2024Robust Multimodal Learning via Representation Decouplingcited by 17
  15. 2022Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalitiescited by 16
  16. 2024HyperFusion: A Hypernetwork Approach to Multimodal Integration of Tabular and Medical Imaging Data for Predictive Modelingcited by 15
  17. 2025STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classificationcited by 12
  18. 2025TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learningcited by 12
  19. 2026MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learningcited by 10
  20. 2025GLCP: Global-to-Local Connectivity Preservation for Tubular Structure Segmentationcited by 9
  21. 2025Unleashing the Power of Image-Tabular Self-Supervised Learning via Breaking Cross-Tabular Barrierscited by 5
  22. 2024Predicting Stroke through Retinal Graphs and Multimodal Self-supervised Learningcited by 2
  23. 2024EPE-P: Evidence-based Parameter-efficient Prompting for Multimodal Learning with Missing Modalitiescited by 2

Built on by

Not cited yet.