Sublanguage: A Serious Issue Affects Pretrained Models in Legal Domain
Not on the map. Why?
Abstract (arXiv)
Legal English is a sublanguage that is important for everyone but not for everyone to understand. Pretrained models have become best practices among current deep learning approaches for different problems. It would be a waste or even a danger if these models were applied in practice without knowledge of the sublanguage of the law. In this paper, we raise the issue and propose a trivial solution by introducing BERTLaw a legal sublanguage pretrained model. The paper's experiments demonstrate the superior effectiveness of the method compared to the baseline pretrained model
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 19 references that are in the map. First posted on arXiv 15 Apr 2021.
Builds on
8 of 19 references.
References · 8
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2018Deep contextualized word representationscited by 2,290
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
Built on by
Not cited yet.