# Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification

arXiv: https://arxiv.org/abs/2311.16650 · HTML: https://mlsanity.com/paper/2311.16650/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 30 of 31)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples](https://mlsanity.com/paper/1704.07433/) (2017)
- [Feature Space Augmentation for Long-Tailed Data](https://mlsanity.com/paper/2008.03673/) (2020)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [BAGAN: Data Augmentation with Balancing GAN](https://mlsanity.com/paper/1803.09655/) (2018)
- [Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding](https://mlsanity.com/paper/2203.01515/) (2022)
- [Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification](https://mlsanity.com/paper/2203.03825/) (2022)
- [T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction](https://mlsanity.com/paper/2211.16887/) (2022)
- [Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding](https://mlsanity.com/paper/2210.03304/) (2022)
- [Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution](https://mlsanity.com/paper/2109.04712/) (2021)
- [Inferring Degrees from Incomplete Networks and Nonlinear Dynamics](https://mlsanity.com/paper/2004.10546/) (2020)
- [HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification](https://mlsanity.com/paper/2204.13413/) (2022)
- [Self-paced Ensemble for Highly Imbalanced Massive Data Classification](https://mlsanity.com/paper/1909.03500/) (2019)
- [Pure Noise to the Rescue of Insufficient Data: Improving Imbalanced Classification by Training on Random Noise Images](https://mlsanity.com/paper/2112.08810/) (2021)
- [Exploiting Global and Local Hierarchies for Hierarchical Text Classification](https://mlsanity.com/paper/2205.02613/) (2022)
- [Condensed Memory Networks for Clinical Diagnostic Inferencing](https://mlsanity.com/paper/1612.01848/) (2016)
- [Making Pretrained Language Models Good Long-tailed Learners](https://mlsanity.com/paper/2205.05461/) (2022)
- [Modeling Diagnostic Label Correlation for Automatic ICD Coding](https://mlsanity.com/paper/2106.12800/) (2021)

## Built on by (most cited first, 1 of 1)

- [TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model](https://mlsanity.com/paper/2404.01273/) (2024)

## Abstract (arXiv)

Deep learning approaches exhibit promising performances on various text tasks. However, they are still struggling on medical text classification since samples are often extremely imbalanced and scarce. Different from existing mainstream approaches that focus on supplementary semantics with external medical information, this paper aims to rethink the data challenges in medical texts and present a novel framework-agnostic algorithm called Text2Tree that only utilizes internal label hierarchy in training deep learning models. We embed the ICD code tree structure of labels into cascade attention modules for learning hierarchy-aware label representations. Two new learning schemes, Similarity Surrogate Learning (SSL) and Dissimilarity Mixup Learning (DML), are devised to boost text classification by reusing and distinguishing samples of other labels following the label representation hierarchy, respectively. Experiments on authoritative public datasets and real-world medical records show that our approach stably achieves superior performances over classical and advanced imbalanced classification methods.