# Unsupervised Law Article Mining based on Deep Pre-Trained Language Representation Models with Application to the Italian Civil Code

arXiv: https://arxiv.org/abs/2112.03033 · HTML: https://mlsanity.com/paper/2112.03033/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [LEGAL-BERT: The Muppets straight out of Law School](https://mlsanity.com/paper/2010.02559/) (2020)
- [Comparative Study of CNN and RNN for Natural Language Processing](https://mlsanity.com/paper/1702.01923/) (2017)
- [Learning to Predict Charges for Criminal Cases with Legal Basis](https://mlsanity.com/paper/1707.09168/) (2017)
- [Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network](https://mlsanity.com/paper/1905.03969/) (2019)
- [Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions](https://mlsanity.com/paper/1802.08504/) (2018)
- [Exploring the Use of Text Classification in the Legal Domain](https://mlsanity.com/paper/1710.09306/) (2017)
- [Extreme Multi-Label Legal Text Classification: A case study in EU Legislation](https://mlsanity.com/paper/1905.10892/) (2019)
- [Automatic Judgment Prediction via Legal Reading Comprehension](https://mlsanity.com/paper/1809.06537/) (2018)
- [Obligation and Prohibition Extraction Using Hierarchical RNNs](https://mlsanity.com/paper/1805.03871/) (2018)
- [Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network](https://mlsanity.com/paper/1703.05320/) (2017)
- [JNLP Team: Deep Learning Approaches for Legal Processing Tasks in COLIEE 2021](https://mlsanity.com/paper/2106.13405/) (2021)

## Built on by (most cited first, 6 of 6)

- [Bringing order into the realm of Transformer-based language models for artificial intelligence and law](https://mlsanity.com/paper/2308.05502/) (2023)
- [Legal Documents Drafting with Fine-Tuned Pre-Trained Large Language Model](https://mlsanity.com/paper/2406.04202/) (2024)
- [A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building Codes](https://mlsanity.com/paper/2309.14374/) (2023)
- [Automatic explanation of the classification of Spanish legal judgments in jurisdiction-dependent law categories with tree estimators](https://mlsanity.com/paper/2404.00437/) (2024)
- [A Two-Stage Architecture for NDA Analysis: LLM-based Segmentation and Transformer-based Clause Classification](https://mlsanity.com/paper/2603.09990/) (2026)
- [Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights](https://mlsanity.com/paper/2610.00087/) (2026)

## Abstract (arXiv)

Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine-tuning an Italian pre-trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few-shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single-label as well as multi-label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep-learning text classifiers and a few-shot learner conceived for an attribute-aware prediction task.