mlsanityData to 14 Oct 2026

NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation

arXiv 2609.17699 ↗First posted 15 Sep 2026cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present NeMo Data Designer (NDD), an open-source, general-purpose framework for multi-modal synthetic data generation (SDG). Designed to be intuitive to use, NDD provides a declarative configuration format in which human and/or agent users define each dataset column, with column types spanning text, code, structured outputs, images, embeddings, and statistical samplers that are explicitly configured to steer dataset diversity. Additional column types and functionality can be introduced using the framework's flexible plugin system. NDD's configuration is an inspectable artifact, supporting workflow sharing and reproducibility. SDG is an inherently iterative process. NDD therefore builds a preview-and-revision loop into its core workflow, allowing users to generate and inspect a small number of records, refine the specification, and rerun generation at full scale. At runtime, NDD resolves dependencies, schedules calls to user-provided model endpoints, and retries failed requests. We describe NDD's architecture and programming model and present case studies spanning structured, agentic, multimodal, and domain-specialized tasks, including datasets used in Nemotron model development and in production enterprise deployments.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

26 of 52 references.

References · 26

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  4. 2023Textbooks Are All You Needcited by 569
  5. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  6. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  7. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  8. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  9. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  10. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  11. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  12. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 161
14 more
  1. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
  2. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  3. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 104
  4. 2024Strong Model Collapsecited by 65
  5. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  6. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 45
  7. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42
  8. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 38
  9. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
  10. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfallscited by 26
  11. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 25
  12. 2026AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversationscited by 20
  13. 2026ProactBench: Beyond What The User Asked Forcited by 4
  14. 2026PICon: A Multi-Turn Interrogation Framework for Evaluating Persona Agent Consistencycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MedEvo: Medical Reasoning Evolution through Knowledge Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning-Driven Synthetic Data Generation and Evaluation
  2. 2026Synthesis Without Training: An Inference-Only Pipeline for Tabular, Temporal, and Relational Synthetic Data
  3. 2026DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis
  4. 2025FASTGEN: Fast and Cost-Effective Synthetic Tabular Data Generation with LLMs
  5. 2026Efficient Embedding-based Synthetic Data Generation for Complex Reasoning Tasks
  6. 2024Comprehensive Exploration of Synthetic Data Generation: A Survey
  7. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
  8. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models