# Meltemi: The first open Large Language Model for Greek

arXiv: https://arxiv.org/abs/2407.20743 · HTML: https://mlsanity.com/paper/2407.20743/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages](https://mlsanity.com/paper/2309.09400/) (2023)
- [MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer](https://mlsanity.com/paper/2109.00904/) (2021)
- [A New Massive Multilingual Dataset for High-Performance Language Technologies](https://mlsanity.com/paper/2403.14009/) (2024)
- [RakutenAI-7B: Extending Large Language Models for Japanese](https://mlsanity.com/paper/2403.15484/) (2024)

## Built on by (most cited first, 17 of 17)

- [Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance](https://mlsanity.com/paper/2502.18772/) (2025)
- [GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations](https://mlsanity.com/paper/2505.17267/) (2025)
- [Krikri: Advancing Open Large Language Models for Greek](https://mlsanity.com/paper/2505.13772/) (2025)
- [BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data](https://mlsanity.com/paper/2510.10159/) (2025)
- [GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek](https://mlsanity.com/paper/2602.05150/) (2026)
- [Kuwain 1.5B: An Arabic SLM via Language Injection](https://mlsanity.com/paper/2504.15120/) (2025)
- [CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning](https://mlsanity.com/paper/2506.17818/) (2025)
- [Evaluating Monolingual and Multilingual Large Language Models for Greek Question Answering: The DemosQA Benchmark](https://mlsanity.com/paper/2602.16811/) (2026)
- [Open or Closed LLM for Lesser-Resourced Languages? Lessons from Greek](https://mlsanity.com/paper/2501.12826/) (2025)
- [Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation](https://mlsanity.com/paper/2510.20002/) (2025)
- [Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models](https://mlsanity.com/paper/2605.18504/) (2026)
- [MORFES: A Benchmark for Productive Inflectional Competence in Modern Greek](https://mlsanity.com/paper/2607.28274/) (2026)
- [GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek](https://mlsanity.com/paper/2412.08520/) (2024)
- [SnakModel: Lessons Learned from Training an Open Danish Large Language Model](https://mlsanity.com/paper/2412.12956/) (2024)
- [Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models](https://mlsanity.com/paper/2605.01870/) (2026)
- [The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs](https://mlsanity.com/paper/2606.07422/) (2026)
- [Tool-calling retrieval versus vector RAG for a small Greek--English knowledge base: accuracy and robustness to how users type Greek](https://mlsanity.com/paper/2610.08205/) (2026)

## Abstract (arXiv)

We describe the development and capabilities of Meltemi 7B, the first open Large Language Model for the Greek language. Meltemi 7B has 7 billion parameters and is trained on a 40 billion token Greek corpus. For the development of Meltemi 7B, we adapt Mistral, by continuous pretraining on the Greek Corpus. Meltemi 7B contains up-to-date information up to September 2023. Furthermore, we have translated and curated a Greek instruction corpus, which has been used for the instruction-tuning of a chat model, named Meltemi 7B Instruct. Special care has been given to the alignment and the removal of toxic content for the Meltemi 7B Instruct. The developed models are evaluated on a broad set of collected evaluation corpora, and examples of prompts and responses are presented. Both Meltemi 7B and Meltemi 7B Instruct are available at https://huggingface.co/ilsp under the Apache 2.0 license.