# MATCH: Metadata-Aware Text Classification in A Large Hierarchy

arXiv: https://arxiv.org/abs/2102.07349 · HTML: https://mlsanity.com/paper/2102.07349/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 70 references that are in the map.
- First posted on arXiv 15 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks](https://mlsanity.com/paper/1508.00200/) (2015)
- [Star-Transformer](https://mlsanity.com/paper/1902.09113/) (2019)
- [DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification](https://mlsanity.com/paper/1609.02521/) (2016)
- [Spherical Text Embedding](https://mlsanity.com/paper/1911.01196/) (2019)
- [Hierarchical Text Classification with Reinforced Label Assignment](https://mlsanity.com/paper/1908.10419/) (2019)
- [A Web-scale system for scientific knowledge exploration](https://mlsanity.com/paper/1805.12216/) (2018)
- [Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification](https://mlsanity.com/paper/1909.04176/) (2019)
- [CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases](https://mlsanity.com/paper/1610.08763/) (2016)
- [LSHTC: A Benchmark for Large-Scale Text Classification](https://mlsanity.com/paper/1503.08581/) (2015)
- [Octet: Online Catalog Taxonomy Enrichment with Self-Supervision](https://mlsanity.com/paper/2006.10276/) (2020)
- [HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories](https://mlsanity.com/paper/1910.07115/) (2019)
- [Hierarchical Metadata-Aware Document Categorization under Weak Supervision](https://mlsanity.com/paper/2010.13556/) (2020)
- [Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection](https://mlsanity.com/paper/1612.04460/) (2016)
- [A Century of Science: Globalization of Scientific Collaborations, Citations, and Innovations](https://mlsanity.com/paper/1704.05150/) (2017)
- [Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity](https://mlsanity.com/paper/1909.01584/) (2019)
- [Minimally Supervised Categorization of Text with Metadata](https://mlsanity.com/paper/2005.00624/) (2020)
- [Categorical Metadata Representation for Customized Text Classification](https://mlsanity.com/paper/1902.05196/) (2019)

## Built on by (most cited first, 20 of 20)

- [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023)
- [TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision](https://mlsanity.com/paper/2403.00165/) (2024)
- [Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks](https://mlsanity.com/paper/2205.10282/) (2022)
- [OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services](https://mlsanity.com/paper/2103.02410/) (2021)
- [The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study](https://mlsanity.com/paper/2302.03341/) (2023)
- [Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding](https://mlsanity.com/paper/2305.14232/) (2023)
- [Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification](https://mlsanity.com/paper/2202.05932/) (2022)
- [MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information](https://mlsanity.com/paper/2111.04022/) (2021)
- [Revisiting Hierarchical Text Classification: Inference and Metrics](https://mlsanity.com/paper/2410.01305/) (2024)
- [HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification](https://mlsanity.com/paper/2402.01696/) (2024)
- [Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers](https://mlsanity.com/paper/2306.14003/) (2023)
- [Classification of Goods Using Text Descriptions With Sentences Retrieval](https://mlsanity.com/paper/2111.01663/) (2021)
- [Hierarchical Compression of Text-Rich Graphs via Large Language Models](https://mlsanity.com/paper/2406.11884/) (2024)
- [Explainable Product Classification for Customs](https://mlsanity.com/paper/2311.10922/) (2023)
- [LabelCoRank: Revolutionizing Long Tail Multi-Label Classification with Co-Occurrence Reranking](https://mlsanity.com/paper/2503.07968/) (2025)
- [Many-Class Text Classification with Matching](https://mlsanity.com/paper/2205.11409/) (2022)
- [MessageNet: Message Classification using Natural Language Processing and Meta-data](https://mlsanity.com/paper/2301.01808/) (2023)
- [Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learning](https://mlsanity.com/paper/2412.13809/) (2024)
- [Hierarchical Multi-Label Generation with Probabilistic Level-Constraint](https://mlsanity.com/paper/2505.03775/) (2025)
- [Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approach](https://mlsanity.com/paper/2605.20674/) (2026)

## Abstract (arXiv)

Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set. Commonly, the metadata of the given documents and the hierarchy of the labels are available in real-world applications. However, most existing studies focus on only modeling the text information, with a few attempts to utilize either metadata or hierarchy signals, but not both of them. In this paper, we bridge the gap by formalizing the problem of metadata-aware text classification in a large label hierarchy (e.g., with tens of thousands of labels). To address this problem, we present the MATCH solution -- an end-to-end framework that leverages both metadata and hierarchy information. To incorporate metadata, we pre-train the embeddings of text and metadata in the same space and also leverage the fully-connected attentions to capture the interrelations between them. To leverage the label hierarchy, we propose different ways to regularize the parameters and output probability of each child label by its parents. Extensive experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH over state-of-the-art deep learning baselines.