mlsanityData to 7 Oct 2026

Nemotron-4 15B Technical Report

arXiv 2402.16819 ↗First posted 26 Feb 2024cs.CL cs.AI cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates strong performance when assessed on English, multilingual, and coding tasks: it outperforms all existing similarly-sized open models on 4 out of 7 downstream evaluation areas and achieves competitive performance to the leading open models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best multilingual capabilities of all similarly-sized models, even outperforming models over four times larger and those explicitly specialized for multilingual tasks.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

26 of 34 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2023Qwen Technical Reportcited by 3,273
14 more
  1. 2023Mistral 7Bcited by 3,065
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  9. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  10. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  11. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  12. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  13. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  14. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 12; 2026: 5.

36 papers.

Sort:

By area:

  1. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  2. 2024An Empirical Study of Mamba-based Language Modelscited by 177
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  5. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  6. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 88
  7. 2024MaskLLM: Learnable Semi-Structured Sparsity for Large Language Modelscited by 49
  8. 2024Upcycling Large Language Models into Mixture of Expertscited by 48
  9. 2024Flextron: Many-in-One Flexible Large Language Modelcited by 42
  10. 2024SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domaincited by 39
  11. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  12. 2025Mid-Training of Large Language Models: A Surveycited by 18
24 more
  1. 2024ESPACE: Dimensionality Reduction of Activations for Model Compressioncited by 15
  2. 2024MIND: Math Informed syNthetic Dialogues for Pretraining LLMscited by 14
  3. 2024Zyda-2: a 5 Trillion Token High-Quality Datasetcited by 12
  4. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 12
  5. 2024Data, Data Everywhere: A Guide for Pretraining Dataset Constructioncited by 11
  6. 2024Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understandingcited by 9
  7. 2024Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuningcited by 7
  8. 2025LO-BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inferencecited by 6
  9. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancementcited by 5
  10. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Modelscited by 5
  11. 2025PARAM-1 BharatGen 2.9B Modelcited by 5
  12. 2024Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methodscited by 4
  13. 2024Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelismcited by 4
  14. 2025PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computingcited by 4
  15. 2026Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Lawscited by 4
  16. 2026Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Languagecited by 4
  17. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  18. 2025K2-V2: A 360-Open, Reasoning-Enhanced LLMcited by 3
  19. 2025Mapping the Italian Telegram Ecosystem: Communities, Toxicity, and Hate Speechcited by 2
  20. 2025Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insightscited by 2
  21. 20266G Needs Agents: Toward Agentic AI-Native Networks for Autonomous Intelligencecited by 2
  22. 2026On the Policy Gradient Foundations of Group Relative Policy Optimization: Credit Assignment, Gradient Sparsity, and Rank Collapsecited by 2
  23. 2024Effective Large Language Model Debugging with Best-first Tree Searchcited by 1
  24. 2026The System Prompt Illusion: How Instruction Preambles Modify Computation in Language Modelscited by 0