mlsanityData to 7 Oct 2026

Corpora deduplication or duplication in Natural Language Processing of few resourced languages ? A case of study: The Mexico's Nahuatl

arXiv 2604.07015 ↗First posted 8 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this article, we seek to answer the following question: could data duplication be useful in Natural Language Processing (NLP) for languages with limited computational resources? In this type of languages (or π-languages), corpora available for training Large Language Models are virtually non-existent. In particular, we will study the impact of corpora expansion in Nawatl, an agglutinative and polysynthetic π-language spoken by over 2 million people, with a large number of dialectal varieties. The aim is to expand the new π-yalli corpus, which contains a limited number of Nawatl texts, by duplicating it in a controlled way. In our experiments, we will use the incremental duplication technique. The aim is to learn embeddings that are well-suited to NLP tasks. Thus, static embeddings were trained and evaluated in a sentence-level semantic similarity task. Our results show a moderate improvement in performance when using incremental duplication compared to the results obtained using only the corpus without expansion. Furthermore, to our knowledge, this technique has not yet been used in the literature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 40 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

16 of 40 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  7. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  8. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  9. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  10. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  11. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  12. 2020On the importance of pre-training data volume for compact language modelscited by 21
4 more
  1. 2016Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networkscited by 14
  2. 2023Distributional Data Augmentation Methods for Low Resource Languagecited by 4
  3. 2025Two CFG Nahuatl for automatic corpora expansioncited by 1
  4. 2026Classifying several dialectal Nawatl varietiescited by 1

Built on by

Not cited yet.