mlsanityData to 13 Oct 2026

The Art of Asking: Multilingual Prompt Optimization for Synthetic Data

arXiv 2510.19806 ↗First posted 22 Oct 2025cs.CLcited by 5

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Synthetic data has become a cornerstone for scaling large language models, yet its multilingual use remains bottlenecked by translation-based prompts. This strategy inherits English-centric framing and style and neglects cultural dimensions, ultimately constraining model generalization. We argue that the overlooked prompt space-the very inputs that define training distributions-offers a more powerful lever for improving multilingual performance. We introduce a lightweight framework for prompt-space optimization, where translated prompts are systematically transformed for Naturalness, Cultural Adaptation, and Difficulty Enhancement. Using an off-the-shelf multilingual LLM, we apply these transformations to prompts for 12 languages spanning 7 families. Under identical data conditions, our approaches achieve substantial and consistent downstream improvements over the translation-only baseline: +4.7% on Global-MMLU accuracy, +2.4% on Flores XCometXL and +35.3% wins in preferences on mArenaHard. We establish prompt-space optimization as a simple yet powerful paradigm for building multilingual LLMs that are more robust, culturally grounded, and globally capable.

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 27 of its 41 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

27 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey14 Jun

Other references · 26

  1. 2025s1: Simple test-time scalingcited by 1,322
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,302
  3. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,120
  4. 2023LIMA: Less Is More for Alignmentcited by 1,091
  5. 2016Bag of Tricks for Efficient Text Classificationcited by 872
  6. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 525
  7. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 486
  8. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 458
  9. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  10. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  11. 2023Does Writing with Language Models Reduce Content Diversity?cited by 191
  12. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 135
14 more
  1. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 80
  2. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
  3. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  4. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  5. 2024Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scorescited by 50
  6. 2024Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuraciescited by 40
  7. 2024A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelismcited by 33
  8. 2025BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Modelscited by 23
  9. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  10. 2024Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learningcited by 17
  11. 2024EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Modelscited by 16
  12. 2025Lost in Literalism: How Supervised Training Shapes Translationese in LLMscited by 11
  13. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  14. 2025Prompt Engineering: How Prompt Vocabulary affects Domain Knowledgecited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15
  2. 2026The Culture Funnel: You Can't Align What isn't in the Datacited by 2
  3. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  4. 2026Accurate in space, unreliable in time: how LLMs represent national cultural changecited by 0
  5. 2026Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
  2. 2024Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach
  3. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  4. 2024RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
  5. 2025How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting
  6. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
  7. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
  8. 2025EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context