# The Development of a Labelled te reo M=aori-English Bilingual Database for Language Technology

arXiv: https://arxiv.org/abs/2208.09778 · HTML: https://mlsanity.com/paper/2208.09778/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 30 references that are in the map.
- First posted on arXiv 21 Aug 2022.

## Builds on (most cited first, 2 of 2)

- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)

## Built on by (most cited first, 1 of 1)

- [Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societies](https://mlsanity.com/paper/2312.01509/) (2023)

## Abstract (arXiv)

Te reo M=aori (referred to as M=aori), New Zealand's indigenous language, is under-resourced in language technology. M=aori speakers are bilingual, where M=aori is code-switched with English. Unfortunately, there are minimal resources available for M=aori language technology, language detection and code-switch detection between M=aori-English pair. Both English and M=aori use Roman-derived orthography making rule-based systems for detecting language and code-switching restrictive. Most M=aori language detection is done manually by language experts. This research builds a M=aori-English bilingual database of 66,016,807 words with word-level language annotation. The New Zealand Parliament Hansard debates reports were used to build the database. The language labels are assigned using language-specific rules and expert manual annotations. Words with the same spelling, but different meanings, exist for M=aori and English. These words could not be categorised as M=aori or English based on word-level language rules. Hence, manual annotations were necessary. An analysis reporting the various aspects of the database such as metadata, year-wise analysis, frequently occurring words, sentence length and N-grams is also reported. The database developed here is a valuable tool for future language and speech technology development for Aotearoa New Zealand. The methodology followed to label the database can also be followed by other low-resourced language pairs.