# Joint Embedding of Words and Labels for Text Classification

arXiv: https://arxiv.org/abs/1805.04174 · HTML: https://mlsanity.com/paper/1805.04174/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2018.

## Built on by (most cited first, 30 of 69)

- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [BertGCN: Transductive Text Classification by Combining GCN and BERT](https://mlsanity.com/paper/2105.05727/) (2021)
- [A Unified Review of Deep Learning for Automated Medical Coding](https://mlsanity.com/paper/2201.02797/) (2022)
- [Towards BERT-based Automatic ICD Coding: Limitations and Opportunities](https://mlsanity.com/paper/2104.06709/) (2021)
- [Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design](https://mlsanity.com/paper/2106.13058/) (2021)
- [Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines](https://mlsanity.com/paper/2107.10650/) (2021)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction](https://mlsanity.com/paper/2109.04108/) (2021)
- [Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning](https://mlsanity.com/paper/2106.03103/) (2021)
- [TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding](https://mlsanity.com/paper/2104.10652/) (2021)
- [Hierarchical Label-wise Attention Transformer Model for Explainable ICD Coding](https://mlsanity.com/paper/2204.10716/) (2022)
- [AnnoBERT: Effectively Representing Multiple Annotators' Label Choices to Improve Hate Speech Detection](https://mlsanity.com/paper/2212.10405/) (2022)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)
- [Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings](https://mlsanity.com/paper/2102.01156/) (2021)
- [Evaluating Multi-label Classifiers with Noisy Labels](https://mlsanity.com/paper/2102.08427/) (2021)
- [A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries](https://mlsanity.com/paper/2107.10652/) (2021)
- [ICDBigBird: A Contextual Embedding Model for ICD Code Classification](https://mlsanity.com/paper/2204.10408/) (2022)
- [Time-Series JEPA for Predictive Remote Control under Capacity-Limited Networks](https://mlsanity.com/paper/2406.04853/) (2024)
- [InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification](https://mlsanity.com/paper/2109.07319/) (2021)
- [A Multi-Granularity Matching Attention Network for Query Intent Classification in E-commerce Retrieval](https://mlsanity.com/paper/2303.15870/) (2023)
- [Label Anchored Contrastive Learning for Language Understanding](https://mlsanity.com/paper/2205.10227/) (2022)
- ["Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definition](https://mlsanity.com/paper/2402.03221/) (2024)
- [A General Knowledge Injection Framework for ICD Coding](https://mlsanity.com/paper/2505.18708/) (2025)
- [HiCu: Leveraging Hierarchy for Curriculum Learning in Automated ICD Coding](https://mlsanity.com/paper/2208.02301/) (2022)
- [SemSup-XC: Semantic Supervision for Zero and Few-shot Extreme Classification](https://mlsanity.com/paper/2301.11309/) (2023)
- [Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records](https://mlsanity.com/paper/2201.06779/) (2022)
- [A Semi-supervised Multi-channel Graph Convolutional Network for Query Classification in E-commerce](https://mlsanity.com/paper/2408.01928/) (2024)
- [Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review](https://mlsanity.com/paper/2412.18043/) (2024)
- [UniMoCo: Unified Modality Completion for Robust Multi-Modal Embeddings](https://mlsanity.com/paper/2505.11815/) (2025)
- [Hierarchical BERT for Medical Document Understanding](https://mlsanity.com/paper/2204.09600/) (2022)

## Abstract (arXiv)

Word embeddings are effective intermediate representations for capturing semantic regularities between words, when learning the representations of text sequences. We propose to view text classification as a label-word joint embedding problem: each label is embedded in the same space with the word vectors. We introduce an attention framework that measures the compatibility of embeddings between text sequences and labels. The attention is learned on a training set of labeled samples to ensure that, given a text sequence, the relevant words are weighted higher than the irrelevant ones. Our method maintains the interpretability of word embeddings, and enjoys a built-in ability to leverage alternative sources of information, in addition to input text sequences. Extensive results on the several large text datasets show that the proposed framework outperforms the state-of-the-art methods by a large margin, in terms of both accuracy and speed.