mlsanityData to 7 Oct 2026

Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation

arXiv 2404.17875 ↗First posted 27 Apr 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Previous graph neural networks (GNNs) usually assume that the graph data is with clean labels for representation learning, but it is not true in real applications. In this paper, we propose a new multi-teacher distillation method based on bi-level optimization (namely BO-NNC), to conduct noisy node classification on the graph data. Specifically, we first employ multiple self-supervised learning methods to train diverse teacher models, and then aggregate their predictions through a teacher weight matrix. Furthermore, we design a new bi-level optimization strategy to dynamically adjust the teacher weight matrix based on the training progress of the student model. Finally, we design a label improvement module to improve the label quality. Extensive experimental results on real datasets show that our method achieves the best results compared to state-of-the-art methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 27 references that are in the map. First posted on arXiv 27 Apr 2024.

Builds on

23 of 27 references.

References · 23

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2017Graph Attention Networkscited by 8,854
  4. 2018Deep Graph Infomaxcited by 1,101
  5. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  6. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  7. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  8. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  9. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  10. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  11. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  12. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
11 more
  1. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
  2. 2020Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Informationcited by 168
  3. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  4. 2020Self-Knowledge Distillation with Progressive Refinement of Targetscited by 106
  5. 2021NRGNN: Learning a Label Noise-Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphscited by 73
  6. 2021A Value-Function-based Interior-point Method for Non-convex Bi-level Optimizationcited by 60
  7. 2022Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminatingcited by 57
  8. 2021Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterioncited by 36
  9. 2021Unified Robust Training for Graph NeuralNetworks against Label Noisecited by 21
  10. 2020Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognitioncited by 6
  11. 2022Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2025Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directionscited by 25
  2. 2024Prompt-based Unifying Inference Attack on Graph Neural Networkscited by 3
  3. 2026Powerful Teachers Matter: Text-Guided Multi-view Knowledge Distillation with Visual Prior Enhancementcited by 0
  4. 2026PaSta: Noisy Node Classification with Partial Label Learningcited by 0