mlsanityData to 14 Oct 2026

Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification

arXiv 2601.16278 ↗First posted 22 Jan 2026cs.CL cs.AI cs.LGcited by 2

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable multilingual capabilities, making them promising tools in both high- and low-resource languages. One particularly valuable use case is generating synthetic samples that can be used to train smaller models in low-resource scenarios where human-labelled data is scarce. In this work, we investigate whether these synthetic data generation capabilities can serve as a form of distillation, producing smaller models that perform on par with or even better than massive LLMs across languages and tasks. To this end, we use a state-of-the-art multilingual LLM to generate synthetic datasets covering 11 languages and 4 classification tasks. These datasets are then used to train smaller models via fine-tuning or instruction tuning, or as synthetic in-context examples for compact LLMs. Our experiments show that even small amounts of synthetic data enable smaller models to outperform the large generator itself, particularly in low-resource languages. Overall, the results suggest that LLMs are best utilised as generators (teachers) rather than classifiers, producing data that empowers smaller and more efficient multilingual models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 29 of its 50 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

29 of 50 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  5. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  6. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  7. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  8. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 187
  9. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  10. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  11. 2023Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluationcited by 136
  12. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
17 more
  1. 2020Training Question Answering Models From Synthetic Datacited by 95
  2. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  3. 2024Do LLMs write like humans? Variation in grammatical and rhetorical stylescited by 62
  4. 2023ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustnesscited by 41
  5. 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
  6. 2023Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learningcited by 31
  7. 2023People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detectioncited by 25
  8. 2023Using GPT-4 to Augment Unbalanced Data for Automatic Scoringcited by 20
  9. 2024Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performancecited by 18
  10. 2024Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentationcited by 17
  11. 2024Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapterscited by 15
  12. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  13. 2021Semi-automatic Generation of Multilingual Datasets for Stance Detection in Twittercited by 10
  14. 2025Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checkingcited by 8
  15. 2024GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledgecited by 7
  16. 2024Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classificationcited by 7
  17. 2025MELABenchv1: Benchmarking Large Language Models against Smaller Fine-Tuned Models for Low-Resource Maltese NLPcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026MKJ at SemEval-2026 Task 9: A Comparative Study of Generalist, Specialist, and Ensemble Strategies for Multilingual Polarizationcited by 0
  2. 2026Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
  2. 2025Scaling Low-Resource MT via Synthetic Data Generation with LLMs
  3. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  4. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  5. 2023Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
  6. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Models
  7. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  8. 2025What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning