# MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information

arXiv: https://arxiv.org/abs/2111.04022 · HTML: https://mlsanity.com/paper/2111.04022/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 7 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Higher-order organization of complex networks](https://mlsanity.com/paper/1612.08447/) (2016)
- [PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks](https://mlsanity.com/paper/1508.00200/) (2015)
- [Star-Transformer](https://mlsanity.com/paper/1902.09113/) (2019)
- [Automated Phrase Mining from Massive Text Corpora](https://mlsanity.com/paper/1702.04457/) (2017)
- [Weakly-Supervised Neural Text Classification](https://mlsanity.com/paper/1809.01478/) (2018)
- [Spherical Text Embedding](https://mlsanity.com/paper/1911.01196/) (2019)
- [X-Class: Text Classification with Extremely Weak Supervision](https://mlsanity.com/paper/2010.12794/) (2020)
- [Text Classification Using Label Names Only: A Language Model Self-Training Approach](https://mlsanity.com/paper/2010.07245/) (2020)
- [Discriminative Topic Mining via Category-Name Guided Text Embedding](https://mlsanity.com/paper/1908.07162/) (2019)
- [MATCH: Metadata-Aware Text Classification in A Large Hierarchy](https://mlsanity.com/paper/2102.07349/) (2021)
- [HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories](https://mlsanity.com/paper/1910.07115/) (2019)
- [Hierarchical Metadata-Aware Document Categorization under Weak Supervision](https://mlsanity.com/paper/2010.13556/) (2020)
- [Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks](https://mlsanity.com/paper/2102.11479/) (2021)
- [Minimally Supervised Categorization of Text with Metadata](https://mlsanity.com/paper/2005.00624/) (2020)
- [Categorical Metadata Representation for Customized Text Classification](https://mlsanity.com/paper/1902.05196/) (2019)
- [Towards Real-Time, Country-Level Location Classification of Worldwide Tweets](https://mlsanity.com/paper/1604.07236/) (2016)

## Built on by (most cited first, 9 of 9)

- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study](https://mlsanity.com/paper/2302.03341/) (2023)
- [Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts](https://mlsanity.com/paper/2212.06002/) (2022)
- [Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification](https://mlsanity.com/paper/2202.05932/) (2022)
- [Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training](https://mlsanity.com/paper/2306.07193/) (2023)
- [Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds](https://mlsanity.com/paper/2205.01845/) (2022)
- [PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training](https://mlsanity.com/paper/2305.13723/) (2023)
- [Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers](https://mlsanity.com/paper/2306.14003/) (2023)
- [RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rules](https://mlsanity.com/paper/2403.02932/) (2024)

## Abstract (arXiv)

We study the problem of weakly supervised text classification, which aims to classify text documents into a set of pre-defined categories with category surface names only and without any annotated training document provided. Most existing classifiers leverage textual information in each document. However, in many domains, documents are accompanied by various types of metadata (e.g., authors, venue, and year of a research paper). These metadata and their combinations may serve as strong category indicators in addition to textual contents. In this paper, we explore the potential of using metadata to help weakly supervised text classification. To be specific, we model the relationships between documents and metadata via a heterogeneous information network. To effectively capture higher-order structures in the network, we use motifs to describe metadata combinations. We propose a novel framework, named MotifClass, which (1) selects category-indicative motif instances, (2) retrieves and generates pseudo-labeled training samples based on category names and indicative motif instances, and (3) trains a text classifier using the pseudo training data. Extensive experiments on real-world datasets demonstrate the superior performance of MotifClass to existing weakly supervised text classification approaches. Further analysis shows the benefit of considering higher-order metadata information in our framework.