# Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification

arXiv: https://arxiv.org/abs/2203.03825 · HTML: https://mlsanity.com/paper/2203.03825/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://mlsanity.com/paper/2102.08473/) (2021)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations](https://mlsanity.com/paper/2005.04364/) (2020)
- [CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding](https://mlsanity.com/paper/2107.00440/) (2021)
- [Hierarchical Text Classification with Reinforced Label Assignment](https://mlsanity.com/paper/1908.10419/) (2019)
- [Effective Use of Word Order for Text Categorization with Convolutional Neural Networks](https://mlsanity.com/paper/1412.1058/) (2014)
- [Improved Text Classification via Contrastive Adversarial Training](https://mlsanity.com/paper/2107.10137/) (2021)
- [Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification](https://mlsanity.com/paper/1909.04176/) (2019)
- [Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/1906.04898/) (2019)
- [Hyperbolic Interaction Model For Hierarchical Multi-Label Classification](https://mlsanity.com/paper/1905.10802/) (2019)
- [HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization](https://mlsanity.com/paper/2104.05220/) (2021)
- [LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Network](https://mlsanity.com/paper/2009.10938/) (2020)

## Built on by (most cited first, 30 of 52)

- [HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification](https://mlsanity.com/paper/2204.13413/) (2022)
- [The ACL OCL Corpus: Advancing Open Science in Computational Linguistics](https://mlsanity.com/paper/2305.14996/) (2023)
- [HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification](https://mlsanity.com/paper/2305.15182/) (2023)
- [Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification](https://mlsanity.com/paper/2406.17534/) (2024)
- [Exploiting Global and Local Hierarchies for Hierarchical Text Classification](https://mlsanity.com/paper/2205.02613/) (2022)
- [Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification](https://mlsanity.com/paper/2305.16885/) (2023)
- [Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2402.15080/) (2024)
- [Why Misinformation is Created? Detecting them by Integrating Intent Features](https://mlsanity.com/paper/2407.19196/) (2024)
- [Multi-stage Retrieve and Re-rank Model for Automatic Medical Coding Recommendation](https://mlsanity.com/paper/2405.19093/) (2024)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classification](https://mlsanity.com/paper/2305.15282/) (2023)
- [Revisiting Hierarchical Text Classification: Inference and Metrics](https://mlsanity.com/paper/2410.01305/) (2024)
- [The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition](https://mlsanity.com/paper/2505.24840/) (2025)
- [HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification](https://mlsanity.com/paper/2402.01696/) (2024)
- [Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training](https://mlsanity.com/paper/2301.07558/) (2023)
- [Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification](https://mlsanity.com/paper/2310.05128/) (2023)
- [Chain-of-Thought Tuning: Masked Language Models can also Think Step By Step in Natural Language Understanding](https://mlsanity.com/paper/2310.11721/) (2023)
- [Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification](https://mlsanity.com/paper/2404.08720/) (2024)
- [Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models](https://mlsanity.com/paper/2603.00431/) (2026)
- [Learning Label Hierarchy with Supervised Contrastive Learning](https://mlsanity.com/paper/2402.00232/) (2024)
- [Utilizing Local Hierarchy with Adversarial Training for Hierarchical Text Classification](https://mlsanity.com/paper/2402.18825/) (2024)
- [Co-guiding for Multi-intent Spoken Language Understanding](https://mlsanity.com/paper/2312.03716/) (2023)
- [FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-scale Cloud Systems](https://mlsanity.com/paper/2402.17583/) (2024)
- [Hierarchical Query Classification in E-commerce Search](https://mlsanity.com/paper/2403.06021/) (2024)
- [Domain-Hierarchy Adaptation via Chain of Iterative Reasoning for Few-shot Hierarchical Text Classification](https://mlsanity.com/paper/2407.08959/) (2024)
- [Your Extreme Multi-label Classifier is Secretly a Hierarchical Text Classifier for Free](https://mlsanity.com/paper/2411.13687/) (2024)
- [Hierarchical Text Classification with LLM-Refined Taxonomies](https://mlsanity.com/paper/2601.18375/) (2026)
- [Are We Really Making Much Progress in Text Classification? A Comparative Review](https://mlsanity.com/paper/2204.03954/) (2022)
- [Seeded Hierarchical Clustering for Expert-Crafted Taxonomies](https://mlsanity.com/paper/2205.11602/) (2022)
- [Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification](https://mlsanity.com/paper/2311.16650/) (2023)

## Abstract (arXiv)

Hierarchical text classification is a challenging subtask of multi-label classification due to its complex label hierarchy. Existing methods encode text and label hierarchy separately and mix their representations for classification, where the hierarchy remains unchanged for all input text. Instead of modeling them separately, in this work, we propose Hierarchy-guided Contrastive Learning (HGCLR) to directly embed the hierarchy into a text encoder. During training, HGCLR constructs positive samples for input text under the guidance of the label hierarchy. By pulling together the input text and its positive sample, the text encoder can learn to generate the hierarchy-aware text representation independently. Therefore, after training, the HGCLR enhanced text encoder can dispense with the redundant hierarchy. Extensive experiments on three benchmark datasets verify the effectiveness of HGCLR.