mlsanityData to 7 Oct 2026

EngGPT2: Sovereign, Efficient and Open Intelligence

arXiv 2603.16430 ↗First posted 17 Mar 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

EngGPT2-16B-A3B is the latest iteration of Engineering Group's Italian LLM and it's built to be a Sovereign, Efficient and Open model. EngGPT2 is trained on 2.5 trillion tokens - less than Qwen3's 36T or Llama3's 15T - and delivers performance on key benchmarks, including MMLU-Pro, GSM8K, IFEval and HumanEval, comparable to dense models in the 8B-16B range, while requiring one-fifth to half of the inference power, and between one-tenth to one-sixth of the training data and consequent needed training power. Designed as a trained-from-scratch Mixture-of-Experts (MoE) architecture, EngGPT2 features 16 billion parameters with 3 billion active per inference, with expert sizes positioned between those used in GPT-OSS and Qwen3. Approximately 25% of its training corpus consists of Italian-language data, to deliver strong capabilities for European and Italian NLP tasks among models of similar scale. This efficiency aims to position EngGPT2 as a key contributor to the growing portfolio of open-weight European models, combining performance and efficiency with full alignment to the EU AI Act. EngGPT2 is also a single model capable of multiple reasoning modes: non-reasoning, reasoning in Italian or English, and turbo-reasoning (a concise, bullet-point style reasoning available in both languages designed for real-time reasoning use cases). EngGPT2 aims to set a new standard for resource-conscious, high-performance LLMs tailored to European and Italian contexts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

27 of 46 references.

References · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  12. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
15 more
  1. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  2. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  3. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  4. 2016Language Modeling with Gated Convolutional Networkscited by 688
  5. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  6. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  7. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  8. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 189
  9. 2024Are We Done with MMLU?cited by 178
  10. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  11. 2024Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentcited by 46
  12. 2025Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Modelscited by 44
  13. 2025Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initializationcited by 23
  14. 2025Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resourcecited by 7
  15. 2024Accelerated Preference Optimization for Large Language Model Alignmentcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking EngGPT2-16B-A3B against Comparable Italian and International Open-source LLMs
  2. 2025Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
  3. 2025Comparison of Large Language Models for Deployment Requirements
  4. 2024PatentGPT: A Large Language Model for Intellectual Property
  5. 2023GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond
  6. 2024InternLM2 Technical Report
  7. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement
  8. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series