mlsanityData to 7 Oct 2026

From Data to Device: ELMOD An Efficient German-First 2.7B Language Model for Mobile Inference

arXiv 2607.24585 ↗First posted 27 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present ELMOD - Efficient Language Model for On-Device Deployment - a compact (2.7B) German language model designed for efficient inference on resource-constrained hardware. ELMOD was trained on a limited computational budget (55k H100 GPU hours) using exclusively publicly available data. We developed a suite of German-specific data pre-processing, which differ from English-oriented counterparts in their handling of morphological variation, compounding, and orthographic conventions. Furthermore, we introduced a quality filtering and rephrasing step, which increased the instructional quality of the data, improved performance during the annealing phase, and reduced overall compute requirements. Thanks to our architectural model and data choices, including prefiltering, our educational-quality filtering and rephrasal to raise the educational-quality, ELMOD is the strongest performer in its size class (<3B), matching the performance of 7B-parameter models in German.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 41 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

26 of 41 references.

References · 26

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  9. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  12. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
14 more
  1. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 156
  2. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  3. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 112
  4. 2024OLMES: A Standard for Language Model Evaluationscited by 100
  5. 2024NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoningcited by 31
  6. 2024Towards Multilingual LLM Evaluation for European Languagescited by 29
  7. 2024How Does Code Pretraining Affect Language Model Task Performance?cited by 28
  8. 2023Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningcited by 26
  9. 2025Beyond Text Compression: Evaluating Tokenizers Across Scalescited by 11
  10. 2025Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resourcescited by 10
  11. 2024LL"aMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratchcited by 9
  12. 2024Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythiacited by 8
  13. 2025Aleph-Alpha-GermanWeb: Improving German-language LLM pre-training with model-based data curation and synthetic data generationcited by 6
  14. 2025Pre-Training LLMs on a budget: A comparison of three optimizerscited by 4

Built on by

Not cited yet.