# Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network

arXiv: https://arxiv.org/abs/2111.13361 · HTML: https://mlsanity.com/paper/2111.13361/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 22 of 22)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filters](https://mlsanity.com/paper/1705.07664/) (2017)
- [Graph Neural Networks with convolutional ARMA filters](https://mlsanity.com/paper/1901.01343/) (2019)
- [Graph Wavelet Neural Network](https://mlsanity.com/paper/1904.07785/) (2019)
- [Learning Structural Node Embeddings Via Diffusion Wavelets](https://mlsanity.com/paper/1710.10321/) (2017)
- [Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text](https://mlsanity.com/paper/2003.13962/) (2020)
- [Autoregressive Moving Average Graph Filtering](https://mlsanity.com/paper/1602.04436/) (2016)
- [A Unifying Framework in Vector-valued Reproducing Kernel Hilbert Spaces for Manifold Regularization and Co-Regularized Multi-view Learning](https://mlsanity.com/paper/1401.8066/) (2014)
- [Image processing](https://mlsanity.com/paper/1409.2413/) (2014)
- [Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling Fusion](https://mlsanity.com/paper/2011.13572/) (2020)
- [Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval](https://mlsanity.com/paper/1605.09696/) (2016)
- [Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization](https://mlsanity.com/paper/2105.05631/) (2021)
- [Orthogonal Multi-view Analysis by Successive Approximations via Eigenvectors](https://mlsanity.com/paper/2010.01632/) (2020)

## Built on by (most cited first, 6 of 6)

- [TIDE: Time Derivative Diffusion for Deep Learning on Graphs](https://mlsanity.com/paper/2212.02483/) (2022)
- [WATS: Calibrating Graph Neural Networks with Wavelet-Aware Temperature Scaling](https://mlsanity.com/paper/2506.23782/) (2025)
- [Curriculum Learning Meets Weakly Supervised Modality Correlation Learning](https://mlsanity.com/paper/2212.07619/) (2022)
- [Multi-modal biometric authentication: Leveraging shared layer architectures for enhanced security](https://mlsanity.com/paper/2411.02112/) (2024)
- [Smoothed Graph Contrastive Learning via Seamless Proximity Integration](https://mlsanity.com/paper/2402.15270/) (2024)
- [GSpect: Spectral Filtering for Cross-Scale Graph Classification](https://mlsanity.com/paper/2409.00338/) (2024)

## Abstract (arXiv)

Multimodal data provide complementary information of a natural phenomenon by integrating data from various domains with very different statistical properties. Capturing the intra-modality and cross-modality information of multimodal data is the essential capability of multimodal learning methods. The geometry-aware data analysis approaches provide these capabilities by implicitly representing data in various modalities based on their geometric underlying structures. Also, in many applications, data are explicitly defined on an intrinsic geometric structure. Generalizing deep learning methods to the non-Euclidean domains is an emerging research field, which has recently been investigated in many studies. Most of those popular methods are developed for unimodal data. In this paper, a multimodal multi-scaled graph wavelet convolutional network (M-GWCN) is proposed as an end-to-end network. M-GWCN simultaneously finds intra-modality representation by applying the multiscale graph wavelet transform to provide helpful localization properties in the graph domain of each modality, and cross-modality representation by learning permutations that encode correlations among various modalities. M-GWCN is not limited to either the homogeneous modalities with the same number of data, or any prior knowledge indicating correspondences between modalities. Several semi-supervised node classification experiments have been conducted on three popular unimodal explicit graph-based datasets and five multimodal implicit ones. The experimental results indicate the superiority and effectiveness of the proposed methods compared with both spectral graph domain convolutional neural networks and state-of-the-art multimodal methods.