mlsanityData to 14 Oct 2026

An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text

arXiv 2305.05627 ↗First posted 9 May 2023cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets -- two in the legal domain and two in the biomedical domain, each with two levels of label granularity -- and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 17 of its 33 references that are in the map. First posted on arXiv 9 May 2023.

Builds on

17 of 33 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2017Decoupled Weight Decay Regularizationcited by 1,831
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  10. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  11. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  12. 2019Large-Scale Multi-Label Text Classification on EU Legislationcited by 104
5 more
  1. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
  2. 2020An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labelscited by 44
  3. 2021Comprehensive Comparative Study of Multi-Label Classification Methodscited by 38
  4. 2022Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Settingcited by 23
  5. 2021Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answeringcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 10; 2025: 6; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generationcited by 13
  2. 2023Taming the Sigmoid Bottleneck: Provably Argmaxable Sparse Multi-Label Classificationcited by 11
  3. 2023Rather a Nurse than a Physician -- Contrastive Explanations under Investigationcited by 11
  4. 2024LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in Englishcited by 8
  5. 2024Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classificationcited by 4
  6. 2023Retrieval-augmented Multi-label Text Classificationcited by 3
  7. 2023A Multi-Modal Multilingual Benchmark for Document Image Classificationcited by 3
  8. 2024FAIIR: Building Toward A Conversational AI Agent Assistant for Youth Mental Health Service Provisioncited by 3
  9. 2025Large Language Models for Healthcare Text Classification: A Systematic Reviewcited by 3
  10. 2024Entailment-Driven Privacy Policy Classification with LLMscited by 2
  11. 2025Modelling and Classifying the Components of a Literature Reviewcited by 2
  12. 2024The Right Model for the Job: An Evaluation of Legal Multi-Label Classification Baselinescited by 1
11 more
  1. 2024Detecting and Characterising Mobile App Metamorphosis in Google Play Storecited by 1
  2. 2025QUAD-LLM-MLTC: Large Language Models Ensemble Learning for Healthcare Text Multi-Label Classificationcited by 1
  3. 2025Divide, Cache, Conquer: Dichotomic Prompting for Efficient Multi-Label LLM-Based Classificationcited by 1
  4. 2024Multi-Label Bayesian Active Learning with Inter-Label Relationshipscited by 0
  5. 2024Enhancing Persona Classification in Dialogue Systems: A Graph Neural Network Approachcited by 0
  6. 2024Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learningcited by 0
  7. 2025University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detectioncited by 0
  8. 2025Skim-Aware Contrastive Learning for Efficient Document Representationcited by 0
  9. 2026Soft Contextualized Encoder For User Defined Text Classificationcited by 0
  10. 2026Task-Centric Acceleration of Small-Language Modelscited by 0
  11. 2026Discrete Diffusion Language Models Are Training-Free Multi-Label Classifierscited by 0