# Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification

arXiv: https://arxiv.org/abs/2310.05128 · HTML: https://mlsanity.com/paper/2310.05128/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 71 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Representation Degeneration Problem in Training Natural Language Generation Models](https://mlsanity.com/paper/1907.12009/) (2019)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [SGM: Sequence Generation Model for Multi-label Classification](https://mlsanity.com/paper/1806.04822/) (2018)
- [Predicting Factuality of Reporting and Bias of News Media Sources](https://mlsanity.com/paper/1810.01765/) (2018)
- [Joint Embedding of Words and Labels for Text Classification](https://mlsanity.com/paper/1805.04174/) (2018)
- [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022)
- [Efficient Large-Scale Multi-Modal Classification](https://mlsanity.com/paper/1802.02892/) (2018)
- [Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification](https://mlsanity.com/paper/2203.03825/) (2022)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [Hierarchical Text Classification with Reinforced Label Assignment](https://mlsanity.com/paper/1908.10419/) (2019)
- [Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification](https://mlsanity.com/paper/1909.04176/) (2019)
- [DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning](https://mlsanity.com/paper/2207.01328/) (2022)
- [HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification](https://mlsanity.com/paper/2204.13413/) (2022)
- [MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid](https://mlsanity.com/paper/2212.14454/) (2022)
- [CAR: Conceptualization-Augmented Reasoner for Zero-Shot Commonsense Question Answering](https://mlsanity.com/paper/2305.14869/) (2023)
- [HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization](https://mlsanity.com/paper/2104.05220/) (2021)
- [DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2305.03973/) (2023)
- [Facilitating Contrastive Learning of Discourse Relational Senses by Exploiting the Hierarchy of Sense Relations](https://mlsanity.com/paper/2301.02724/) (2023)
- [HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification](https://mlsanity.com/paper/2305.15182/) (2023)

## Built on by (most cited first, 4 of 4)

- [Leveraging Label Semantics and Meta-Label Refinement for Multi-Label Question Classification](https://mlsanity.com/paper/2411.01841/) (2024)
- [EPPC-OASIS: Ontology-Aware Adaptation and Structured Inference Refinement for Electronic Patient-Provider Communication Mining in Secure Messages](https://mlsanity.com/paper/2605.24172/) (2026)
- [Taxon: Hierarchical Tax Code Prediction with Semantically Aligned LLM Expert Guidance](https://mlsanity.com/paper/2601.08418/) (2026)
- [Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning](https://mlsanity.com/paper/2603.15708/) (2026)

## Abstract (arXiv)

Hierarchical multi-label text classification (HMTC) aims at utilizing a label hierarchy in multi-label classification. Recent approaches to HMTC deal with the problem of imposing an over-constrained premise on the output space by using contrastive learning on generated samples in a semi-supervised manner to bring text and label embeddings closer. However, the generation of samples tends to introduce noise as it ignores the correlation between similar samples in the same batch. One solution to this issue is supervised contrastive learning, but it remains an underexplored topic in HMTC due to its complex structured labels. To overcome this challenge, we propose HJCL, a Hierarchy-aware Joint Supervised Contrastive Learning method that bridges the gap between supervised contrastive learning and HMTC. Specifically, we employ both instance-wise and label-wise contrastive learning techniques and carefully construct batches to fulfill the contrastive learning objective. Extensive experiments on four multi-path HMTC datasets demonstrate that HJCL achieves promising results and the effectiveness of Contrastive Learning on HMTC.