mlsanityData to 7 Oct 2026

BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification

arXiv 2210.15225 ↗First posted 27 Oct 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-label Text Classification (MLTC) is the task of categorizing documents into one or more topics. Considering the large volumes of data and varying domains of such tasks, fully supervised learning requires manually fully annotated datasets which is costly and time-consuming. In this paper, we propose BERT-Flow-VAE (BFV), a Weakly-Supervised Multi-Label Text Classification (WSMLTC) model that reduces the need for full supervision. This new model (1) produces BERT sentence embeddings and calibrates them using a flow model, (2) generates an initial topic-document matrix by averaging results of a seeded sparse topic model and a textual entailment model which only require surface name of topics and 4-6 seed words per topic, and (3) adopts a VAE framework to reconstruct the embeddings under the guidance of the topic-document matrix. Finally, (4) it uses the means produced by the encoder model in the VAE architecture as predictions for MLTC. Experimental results on 6 multi-label datasets show that BFV can substantially outperform other baseline WSMLTC models in key metrics and achieve approximately 84% performance of a fully-supervised model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

31 of 47 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2016Layer Normalizationcited by 5,579
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
19 more
  1. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  2. 2018Understanding disentangling in β-VAEcited by 332
  3. 2016Ladder Variational Autoencoderscited by 315
  4. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  5. 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 140
  6. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  7. 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
  8. 2020On the Sentence Embeddings from Pre-trained Language Modelscited by 83
  9. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
  10. 2018Weakly-Supervised Neural Text Classificationcited by 66
  11. 2021Self-Training with Weak Supervisioncited by 49
  12. 2020X-Class: Text Classification with Extremely Weak Supervisioncited by 45
  13. 2020Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Startcited by 44
  14. 2020Denoising Multi-Source Weak Supervision for Neural Text Classificationcited by 42
  15. 2016SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoodscited by 32
  16. 2020Text Classification Using Label Names Only: A Language Model Self-Training Approachcited by 32
  17. 2019Discriminative Topic Mining via Category-Name Guided Text Embeddingcited by 29
  18. 2016Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledgecited by 23
  19. 2018Predicting Movie Genres Based on Plot Summariescited by 4

Built on by

Not cited yet.