mlsanityData to 7 Oct 2026

DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslation

arXiv 2601.17823 ↗First posted 25 Jan 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present DIETA, a small, decoder-only Transformer model with 0.5 billion parameters, specifically designed and trained for Italian-English machine translation. We collect and curate a large parallel corpus consisting of approximately 207 million Italian-English sentence pairs across diverse domains, including parliamentary proceedings, legal texts, web-crawled content, subtitles, news, literature and 352 million back-translated data using pretrained models. Additionally, we create and release a new small-scale evaluation set, consisting of 450 sentences, based on 2025 WikiNews articles, enabling assessment of translation quality on contemporary text. Comprehensive evaluations show that DIETA achieves competitive performance on multiple Italian-English benchmarks, consistently ranking in the second quartile of a 32-system leaderboard and outperforming most other sub-3B models on four out of five test suites. The training script, trained models, curated corpus, and newly introduced evaluation set are made publicly available, facilitating further research and development in specialized Italian-English machine translation. https://github.com/pkasela/DIETA-Machine-Translation

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 49 references that are in the map. First posted on arXiv 25 Jan 2026.

Builds on

28 of 49 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  5. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  6. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  7. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  9. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  10. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  11. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  12. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
16 more
  1. 2020Query-Key Normalization for Transformerscited by 266
  2. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  3. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  4. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  5. 2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Languagecited by 134
  6. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 127
  7. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
  8. 2020RealFormer: Transformer Likes Residual Attentioncited by 62
  9. 2022Democratizing Neural Machine Translation with OPUS-MTcited by 57
  10. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  11. 2023LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Languagecited by 24
  12. 2015Building Subject-aligned Comparable Corpora and Mining it for Truly Parallel Sentence Pairscited by 15
  13. 2024Synthetic Data Generation with Large Language Models for Personalized Community Question Answeringcited by 8
  14. 2024Leveraging Large Language Models for Medical Information Extraction and Query Generationcited by 8
  15. 2023SE-PQA: Personalized Community Question Answeringcited by 7
  16. 2023Cerbero-7B: A Leap Forward in Language-Specific LLMs Through Enhanced Chat Corpus Generation and Evaluationcited by 3

Built on by

Not cited yet.