# An Evaluation Dataset for Legal Word Embedding: A Case Study On Chinese Codex

arXiv: https://arxiv.org/abs/2203.15173 · HTML: https://mlsanity.com/paper/2203.15173/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 14 of 14)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Inducing Relational Knowledge from BERT](https://mlsanity.com/paper/1911.12753/) (2019)
- [Analogical Reasoning on Chinese Morphological and Semantic Relations](https://mlsanity.com/paper/1805.06504/) (2018)
- [Evaluating Word Embedding Models: Methods and Experimental Results](https://mlsanity.com/paper/1901.09785/) (2019)
- [Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning](https://mlsanity.com/paper/1509.01692/) (2015)
- [A Survey of Word Embeddings Evaluation Methods](https://mlsanity.com/paper/1801.09536/) (2018)
- [Specialising Word Vectors for Lexical Entailment](https://mlsanity.com/paper/1710.06371/) (2017)
- [Radical-Enhanced Chinese Character Embedding](https://mlsanity.com/paper/1404.4714/) (2014)
- [Component-Enhanced Chinese Character Embeddings](https://mlsanity.com/paper/1508.06669/) (2015)
- [Learning Chinese Word Representations From Glyphs Of Characters](https://mlsanity.com/paper/1708.04755/) (2017)
- [Context-Dependent Word Representation for Neural Machine Translation](https://mlsanity.com/paper/1607.00578/) (2016)
- [RethinkCWS: Is Chinese Word Segmentation a Solved Task?](https://mlsanity.com/paper/2011.06858/) (2020)
- [Relational Word Embeddings](https://mlsanity.com/paper/1906.01373/) (2019)

## Abstract (arXiv)

Word embedding is a modern distributed word representations approach widely used in many natural language processing tasks. Converting the vocabulary in a legal document into a word embedding model facilitates subjecting legal documents to machine learning, deep learning, and other algorithms and subsequently performing the downstream tasks of natural language processing vis-`a-vis, for instance, document classification, contract review, and machine translation. The most common and practical approach of accuracy evaluation with the word embedding model uses a benchmark set with linguistic rules or the relationship between words to perform analogy reasoning via algebraic calculation. This paper proposes establishing a 1,134 Legal Analogical Reasoning Questions Set (LARQS) from the 2,388 Chinese Codex corpus using five kinds of legal relations, which are then used to evaluate the accuracy of the Chinese word embedding model. Moreover, we discovered that legal relations might be ubiquitous in the word embedding model.