mlsanityData to 7 Oct 2026

Node-wise Filtering in Graph Neural Networks: A Mixture of Experts Approach

arXiv 2406.03464 ↗First posted 5 Jun 2024cs.LGcited by 17

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Graph Neural Networks (GNNs) have proven to be highly effective for node classification tasks across diverse graph structural patterns. Traditionally, GNNs employ a uniform global filter, typically a low-pass filter for homophilic graphs and a high-pass filter for heterophilic graphs. However, real-world graphs often exhibit a complex mix of homophilic and heterophilic patterns, rendering a single global filter approach suboptimal. In this work, we theoretically demonstrate that a global filter optimized for one pattern can adversely affect performance on nodes with differing patterns. To address this, we introduce a novel GNN framework Node-MoE that utilizes a mixture of experts to adaptively select the appropriate filters for different nodes. Extensive experiments demonstrate the effectiveness of Node-MoE on both homophilic and heterophilic graphs.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 52 of its 67 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

52 of 67 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling6 Apr

Other references · 51

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  11. 2019Simplifying Graph Convolutional Networkscited by 1,470
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
38 more
  1. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  2. 2020Simple and Deep Graph Convolutional Networkscited by 857
  3. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  4. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  5. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  6. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  7. 2019Multi-scale Attributed Node Embeddingcited by 543
  8. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
  9. 2019Diffusion Improves Graph Learningcited by 414
  10. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  11. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  12. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  13. 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 260
  14. 2017CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filterscited by 203
  15. 2021BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximationcited by 195
  16. 2019Graph Neural Networks with convolutional ARMA filterscited by 185
  17. 2021Is Homophily a Necessity for Graph Neural Networks?cited by 182
  18. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  19. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  20. 2018Contextual Stochastic Block Modelscited by 134
  21. 2016Community detection in networks: A user guidecited by 105
  22. 2021Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patternscited by 95
  23. 2022Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisitedcited by 87
  24. 2021New Benchmarks for Learning on Non-Homophilous Graphscited by 81
  25. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  26. 2021Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalizationcited by 70
  27. 2022Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyondcited by 60
  28. 2022Towards Understanding Mixture of Experts in Deep Learningcited by 52
  29. 2019Overlapping Community Detection with Graph Neural Networkscited by 45
  30. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  31. 2023Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?cited by 41
  32. 2023PC-Conv: Unifying Homophily and Heterophily with Two-fold Filteringcited by 28
  33. 2017Supervised Community Detection with Line Graph Neural Networkscited by 18
  34. 2022Synthetic Graph Generation to Benchmark Graph Learningcited by 16
  35. 2023LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagationcited by 16
  36. 2022Alternately Optimized Graph Neural Networkscited by 4
  37. 2022Improving Your Graph Neural Networks: A High-Frequency Boostercited by 4
  38. 2023Spectral-Aware Augmentation for Enhanced Graph Representation Learningcited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 7.

17 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  2. 2024One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphscited by 12
  3. 2024Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesiscited by 9
  4. 2024Mixture of Experts for Node Classificationcited by 4
  5. 2025Graph Few-Shot Learning via Adaptive Spectrum Experts and Cross-Set Distribution Calibrationcited by 3
  6. 2025Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarseningcited by 2
  7. 2025Multimodal LLM With Hierarchical Mixture-of-Experts for VQA on 3D Brain MRIcited by 2
  8. 2026Topology-Aware Multiscale Mixture of Experts for Efficient Molecular Property Predictioncited by 1
  9. 2026Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learningcited by 1
  10. 2024LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphscited by 0
  11. 2025Towards Anomaly-Aware Pre-Training and Fine-Tuning for Graph Anomaly Detectioncited by 0
  12. 2025Training Diverse Graph Experts for Ensembles: A Systematic Empirical Studycited by 0
5 more
  1. 2026Mitigating Homophily Disparity in Graph Anomaly Detection: A Scalable and Adaptive Approachcited by 0
  2. 2026Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classificationcited by 0
  3. 2026Closed-Form Node Classification with Exact Graph Unlearningcited by 0
  4. 2026Gate the Filter, Not the Message: Node-Channel Mixtures for Pre-Propagation GNNscited by 0
  5. 2026Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification
  2. 2024Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning
  3. 2024Mixture of Link Predictors on Graphs
  4. 2024Sparse Decomposition of Graph Neural Networks
  5. 2025ReDiSC: A Reparameterized Masked Diffusion Model for Scalable Node Classification with Structured Predictions
  6. 2024Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach
  7. 2024DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
  8. 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization