# Large-Scale Multi-Label Text Classification on EU Legislation

arXiv: https://arxiv.org/abs/1906.02192 · HTML: https://mlsanity.com/paper/1906.02192/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 30 of 104)

- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset](https://mlsanity.com/paper/2207.00220/) (2022)
- [MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer](https://mlsanity.com/paper/2109.00904/) (2021)
- [A Statutory Article Retrieval Dataset in French](https://mlsanity.com/paper/2108.11792/) (2021)
- [EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain](https://mlsanity.com/paper/2210.13448/) (2022)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [Pre-trained Language Models for the Legal Domain: A Case Study on Indian Law](https://mlsanity.com/paper/2209.06049/) (2022)
- [Efficient Classification of Long Documents Using Transformers](https://mlsanity.com/paper/2203.11258/) (2022)
- [An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text](https://mlsanity.com/paper/2305.05627/) (2023)
- [Factoring Statutory Reasoning as Language Understanding Challenges](https://mlsanity.com/paper/2105.07903/) (2021)
- [Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformer](https://mlsanity.com/paper/2211.00974/) (2022)
- [Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training](https://mlsanity.com/paper/2506.01732/) (2025)
- [Multi-granular Legal Topic Classification on Greek Legislation](https://mlsanity.com/paper/2109.15298/) (2021)
- [Extreme Zero-Shot Learning for Extreme Text Classification](https://mlsanity.com/paper/2112.08652/) (2021)
- [Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers](https://mlsanity.com/paper/2307.03539/) (2023)
- [Krikri: Advancing Open Large Language Models for Greek](https://mlsanity.com/paper/2505.13772/) (2025)
- [Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations](https://mlsanity.com/paper/2101.10726/) (2021)
- [Dependency Learning for Legal Judgment Prediction with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/2112.06370/) (2021)
- [Few-shot learning for medical text: A systematic review](https://mlsanity.com/paper/2204.14081/) (2022)
- [Open Vocabulary Extreme Classification Using Generative Models](https://mlsanity.com/paper/2205.05812/) (2022)
- [A Survey on Deep Learning Event Extraction: Approaches and Applications](https://mlsanity.com/paper/2107.02126/) (2021)
- [Recurrent Attention Networks for Long-text Modeling](https://mlsanity.com/paper/2306.06843/) (2023)
- [Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification](https://mlsanity.com/paper/2202.05932/) (2022)
- [LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domain](https://mlsanity.com/paper/2404.02127/) (2024)
- [The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation](https://mlsanity.com/paper/2503.12294/) (2025)
- [Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning](https://mlsanity.com/paper/2104.01666/) (2021)
- [CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/2109.04853/) (2021)
- [SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions](https://mlsanity.com/paper/2506.00643/) (2025)

## Abstract (arXiv)

We consider Large-Scale Multi-Label Text Classification (LMTC) in the legal domain. We release a new dataset of 57k legislative documents from EURLEX, annotated with ~4.3k EUROVOC labels, which is suitable for LMTC, few- and zero-shot learning. Experimenting with several neural classifiers, we show that BIGRUs with label-wise attention perform better than other current state of the art methods. Domain-specific WORD2VEC and context-sensitive ELMO embeddings further improve performance. We also find that considering only particular zones of the documents is sufficient. This allows us to bypass BERT's maximum text length limit and fine-tune BERT, obtaining the best results in all but zero-shot learning cases.