mlsanityData to 7 Oct 2026

Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification

arXiv 2406.17534 ↗First posted 25 Jun 2024cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

Hierarchical text classification (HTC) is an important task with broad applications, while few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely-ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically-similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 36 of its 66 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

36 of 66 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  8. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  9. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
24 more
  1. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  2. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  3. 2022Active Example Selection for In-Context Learningcited by 195
  4. 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 158
  5. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  6. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  7. 2017HDLTex: Hierarchical Deep Learning for Text Classificationcited by 105
  8. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  9. 2022An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labelscited by 88
  10. 2023Mitigating Label Biases for In-context Learningcited by 88
  11. 2022Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classificationcited by 52
  12. 2023TIM: Teaching Large Language Models to Translate with Comparisoncited by 50
  13. 2019Hierarchical Text Classification with Reinforced Label Assignmentcited by 41
  14. 2019Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classificationcited by 35
  15. 2022HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classificationcited by 31
  16. 2023ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrievalcited by 26
  17. 2023ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classificationcited by 17
  18. 2018An Analysis of Hierarchical Text Classification Using Word Embeddingscited by 16
  19. 2020LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Networkcited by 11
  20. 2023Hierarchical Verbalizer for Few-Shot Hierarchical Text Classificationcited by 11
  21. 2023A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classificationcited by 7
  22. 2021Hierarchical Text Classification As Sub-Hierarchy Sequence Generationcited by 6
  23. 2023LADER: Log-Augmented DEnse Retrieval for Biomedical Literature Searchcited by 2
  24. 2023EXnet: Efficient In-context Learning for Data-less Text classificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoningcited by 26
  2. 2024KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Servercited by 11
  3. 2025MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Userscited by 10
  4. 2024Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classificationcited by 6
  5. 2025Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspectivecited by 5
  6. 2025Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answeringcited by 5
  7. 2025RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplarscited by 2
  8. 2025Cross-Document Cross-Lingual NLI via RST-Enhanced Graph Fusion and Interpretability Predictioncited by 2
  9. 2025AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compressioncited by 2
  10. 2026HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategycited by 2
  11. 2025Hierarchical Text Classification Using Black Box Large Language Modelscited by 1
  12. 2025Making Language Model a Hierarchical Classifiercited by 0
4 more
  1. 2026Taxon: Hierarchical Tax Code Prediction with Semantically Aligned LLM Expert Guidancecited by 0
  2. 2026Domain-Adapted Retrieval for In-Context Annotation of Pedagogical Dialogue Actscited by 0
  3. 2026SCHK-HTC: Sibling Contrastive Learning with Hierarchical Knowledge-Aware Prompt Tuning for Hierarchical Text Classificationcited by 0
  4. 2026The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspacescited by 0