# Luth: Efficient French Specialization for Small Language Models and Cross-Lingual Transfer

arXiv: https://arxiv.org/abs/2510.05846 · HTML: https://mlsanity.com/paper/2510.05846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [EuroLLM: Multilingual Language Models for Europe](https://mlsanity.com/paper/2409.16235/) (2024)
- [CroissantLLM: A Truly Bilingual French-English Language Model](https://mlsanity.com/paper/2402.00786/) (2024)
- [Mitigating Catastrophic Forgetting in Language Transfer via Model Merging](https://mlsanity.com/paper/2407.08699/) (2024)
- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [Small Language Models are Good Too: An Empirical Study of Zero-Shot Classification](https://mlsanity.com/paper/2404.11122/) (2024)
- [The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation](https://mlsanity.com/paper/2503.12294/) (2025)
- [PAGnol: An Extra-Large French Generative Model](https://mlsanity.com/paper/2110.08554/) (2021)
- [Pensez: Less Data, Better Reasoning -- Rethinking French LLM](https://mlsanity.com/paper/2503.13661/) (2025)

## Built on by (most cited first, 3 of 3)

- [How to Train Your Long-Context Visual Document Model](https://mlsanity.com/paper/2602.15257/) (2026)
- [CARTE: A Benchmark for Mapping Language Model Knowledge Across France](https://mlsanity.com/paper/2606.01995/) (2026)
- [Internalized Reasoning for Long-Context Visual Document Understanding](https://mlsanity.com/paper/2604.02371/) (2026)

## Abstract (arXiv)

The landscape of Large Language Models remains predominantly English-centric, resulting in a significant performance gap for other major languages, such as French, especially in the context of Small Language Models (SLMs). Existing multilingual models demonstrate considerably lower performance in French compared to English, and research on efficient adaptation methods for French remains limited. To address this, we introduce Luth, a family of French-specialized SLMs: through targeted post-training on curated, high-quality French data, our models outperform all open-source counterparts of comparable size on multiple French benchmarks while retaining their original English capabilities. We further show that strategic model merging enhances performance in both languages, establishing Luth as a new state of the art for French SLMs and a robust baseline for future French-language research.