mlsanityData to 7 Oct 2026

Unsupervised Data Validation Methods for Efficient Model Training

arXiv 2410.07880 ↗First posted 10 Oct 2024cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper investigates the challenges and potential solutions for improving machine learning systems for low-resource languages. State-of-the-art models in natural language processing (NLP), text-to-speech (TTS), speech-to-text (STT), and vision-language models (VLM) rely heavily on large datasets, which are often unavailable for low-resource languages. This research explores key areas such as defining "quality data," developing methods for generating appropriate data and enhancing accessibility to model training. A comprehensive review of current methodologies, including data augmentation, multilingual transfer learning, synthetic data generation, and data selection techniques, highlights both advancements and limitations. Several open research questions are identified, providing a framework for future studies aimed at optimizing data utilization, reducing the required data quantity, and maintaining high-quality model performance. By addressing these challenges, the paper aims to make advanced machine learning models more accessible for low-resource languages, enhancing their utility and impact across various sectors.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

25 of 30 references.

References · 25

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  4. 2024Self-Rewarding Language Modelscited by 654
  5. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  6. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  7. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 230
  8. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  9. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  10. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 90
  11. 2024Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Modelscited by 83
  12. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
13 more
  1. 2024Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnosticcited by 74
  2. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  3. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  4. 2024Compression Represents Intelligence Linearlycited by 56
  5. 2024Does your data spark joy? Performance gains from domain upsampling at the end of trainingcited by 51
  6. 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approachcited by 48
  7. 2023No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Modelscited by 41
  8. 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?cited by 35
  9. 2024Zero-Shot Tokenizer Transfercited by 30
  10. 2024Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneckcited by 19
  11. 2024Scaling Laws for the Value of Individual Data Points in Machine Learningcited by 10
  12. 2024Data Selection for Transfer Unlearningcited by 5
  13. 2024Setting up the Data Printer with Improved English to Ukrainian Machine Translationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
  2. 2024Multilingual Large Language Models: A Systematic Survey
  3. 2025Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages
  4. 2024How to Train Data-Efficient LLMs
  5. 2024Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs
  6. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  7. 2025Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets
  8. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations