# verBERT: Automating Brazilian Case Law Document Multi-label Categorization Using BERT

arXiv: https://arxiv.org/abs/2203.06224 · HTML: https://mlsanity.com/paper/2203.06224/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Large-Scale Multi-Label Text Classification on EU Legislation](https://mlsanity.com/paper/1906.02192/) (2019)
- [Portuguese Named Entity Recognition using BERT-CRF](https://mlsanity.com/paper/1909.10649/) (2019)
- [Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis](https://mlsanity.com/paper/2010.04543/) (2020)

## Built on by (most cited first, 1 of 1)

- [Bringing order into the realm of Transformer-based language models for artificial intelligence and law](https://mlsanity.com/paper/2308.05502/) (2023)

## Abstract (arXiv)

In this work, we carried out a study about the use of attention-based algorithms to automate the categorization of Brazilian case law documents. We used data from the Kollemata Project to produce two distinct datasets with adequate class systems. Then, we implemented a multi-class and multi-label version of BERT and fine-tuned different BERT models with the produced datasets. We evaluated several metrics, adopting the micro-averaged F1-Score as our main metric for which we obtained a performance value of F1-micro=0.72 corresponding to gains of 30 percent points over the tested statistical baseline. In this work, we carried out a study about the use of attention-based algorithms to automate the categorization of Brazilian case law documents. We used data from the textitKollemata Project to produce two distinct datasets with adequate class systems. Then, we implemented a multi-class and multi-label version of BERT and fine-tuned different BERT models with the produced datasets. We evaluated several metrics, adopting the micro-averaged F1-Score as our main metric for which we obtained a performance value of langle F1 ranglemicro=0.72 corresponding to gains of 30 percent points over the tested statistical baseline.