mlsanityData to 14 Oct 2026

Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

arXiv 2606.14172 ↗First posted 12 Jun 2026cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

46 of 59 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2017Graph Attention Networkscited by 8,923
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  10. 2017Inductive Representation Learning on Large Graphscited by 6,320
  11. 2018How Powerful are Graph Neural Networks?cited by 4,393
  12. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
34 more
  1. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  2. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  3. 2018Deep Graph Infomaxcited by 1,106
  4. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  5. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  6. 2020Simple and Deep Graph Convolutional Networkscited by 860
  7. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
  8. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  9. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
  10. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
  11. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
  12. 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
  13. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  14. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  15. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  16. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
  17. 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 132
  18. 2020Graph Structured Network for Image-Text Matchingcited by 93
  19. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  20. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  21. 2019Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrievalcited by 66
  22. 2022Multimodal learning with graphscited by 54
  23. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 49
  24. 2019Universal Graph Transformer Self-Attention Networkscited by 41
  25. 2023LGMRec: Local and Global Graph Learning for Multimodal Recommendationcited by 38
  26. 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 29
  27. 2023Multimodal Graph Learning for Generative Taskscited by 27
  28. 2024Learning on Multimodal Graphs: A Surveycited by 22
  29. 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
  30. 2025Graph4MM: Weaving Multimodal Learning with Structural Informationcited by 17
  31. 2025Cross-Contrastive Clustering for Multimodal Attributed Graphs with Dual Graph Filteringcited by 14
  32. 2026OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graphcited by 13
  33. 2021Multiplex Graph Networks for Multimodal Brain Network Analysiscited by 7
  34. 2024Async Learned User Embeddings for Ads Delivery Optimizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.