Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs
Not on the map. Why?
Abstract (arXiv)
Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
46 of 59 references.
References · 46
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2017Graph Attention Networkscited by 8,923
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
34 more
- 2020Graph Contrastive Learning with Augmentationscited by 1,208
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2018Deep Graph Infomaxcited by 1,106
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
- 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
- 2020Simple and Deep Graph Convolutional Networkscited by 860
- 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
- 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
- 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
- 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
- 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
- 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
- 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
- 2021Training Graph Neural Networks with 1000 Layerscited by 169
- 2022Revisiting Heterophily For Graph Neural Networkscited by 169
- 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
- 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 132
- 2020Graph Structured Network for Image-Text Matchingcited by 93
- 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
- 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
- 2019Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrievalcited by 66
- 2022Multimodal learning with graphscited by 54
- 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 49
- 2019Universal Graph Transformer Self-Attention Networkscited by 41
- 2023LGMRec: Local and Global Graph Learning for Multimodal Recommendationcited by 38
- 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 29
- 2023Multimodal Graph Learning for Generative Taskscited by 27
- 2024Learning on Multimodal Graphs: A Surveycited by 22
- 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
- 2025Graph4MM: Weaving Multimodal Learning with Structural Informationcited by 17
- 2025Cross-Contrastive Clustering for Multimodal Attributed Graphs with Dual Graph Filteringcited by 14
- 2026OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graphcited by 13
- 2021Multiplex Graph Networks for Multimodal Brain Network Analysiscited by 7
- 2024Async Learned User Embeddings for Ads Delivery Optimizationcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.