mlsanityData to 7 Oct 2026

FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs

arXiv 2301.10481 ↗First posted 25 Jan 2023cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present FewShotTextGCN, a novel method designed to effectively utilize the properties of word-document graphs for improved learning in low-resource settings. We introduce K-hop Neighbourhood Regularization, a regularizer for heterogeneous graphs, and show that it stabilizes and improves learning when only a few training samples are available. We furthermore propose a simplification in the graph-construction method, which results in a graph that is ~7 times less dense and yields better performance in little-resource settings while performing on par with the state of the art in high-resource settings. Finally, we introduce a new variant of Adaptive Pseudo-Labeling tailored for word-document graphs. When using as little as 20 samples for training, we outperform a strong TextGCN baseline with 17% in absolute accuracy on average over eight languages. We demonstrate that our method can be applied to document classification without any language model pretraining on a wide range of typologically diverse languages while performing on par with large pretrained language models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 25 Jan 2023.

Builds on

29 of 39 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  9. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  12. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
17 more
  1. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  2. 2018Graph Convolutional Networks for Text Classificationcited by 345
  3. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  4. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  5. 2015A Survey of Heterogeneous Information Network Analysiscited by 109
  6. 2020Gradient Centralization: A New Optimization Technique for Deep Neural Networkscited by 82
  7. 2021BertGCN: Transductive Text Classification by Combining GCN and BERTcited by 74
  8. 2020Be More with Less: Hypergraph Attention Networks for Inductive Text Classificationcited by 73
  9. 2020Tensor Graph Convolutional Networks for Text Classificationcited by 49
  10. 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
  11. 2014Effective Use of Word Order for Text Categorization with Convolutional Neural Networkscited by 39
  12. 2019Message Passing Attention Networks for Document Understandingcited by 23
  13. 2021Specializing Multilingual Language Models: An Empirical Studycited by 21
  14. 2020fugashi, a Tool for Tokenizing Japanese in Pythoncited by 17
  15. 2021Cross-lingual Text Classification with Heterogeneous Graph Neural Networkcited by 12
  16. 2019A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddingscited by 4
  17. 2021Graph-based Label Propagation for Semi-Supervised Speaker Identificationcited by 4

Built on by

Not cited yet.