# PLLuM: A Family of Polish Large Language Models

arXiv: https://arxiv.org/abs/2511.03823 · HTML: https://mlsanity.com/paper/2511.03823/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2025.

## Built on by (most cited first, 8 of 8)

- [Long-Context Encoder Models for Polish Language Understanding](https://mlsanity.com/paper/2603.12191/) (2026)
- [The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection](https://mlsanity.com/paper/2606.03305/) (2026)
- [What properties of reasoning supervision are associated with improved downstream model quality?](https://mlsanity.com/paper/2605.13290/) (2026)
- [Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering](https://mlsanity.com/paper/2607.13568/) (2026)
- [From Zero to Hero: An Open LLM Ecosystem for Armenian](https://mlsanity.com/paper/2609.03350/) (2026)
- [The Limits of BPE Tokenization in Polish: Segmentation-Flexional Forms, Grammatical Anchoring, and First-Person Stability in Inflectional Language Models](https://mlsanity.com/paper/2609.17553/) (2026)
- [Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bit](https://mlsanity.com/paper/2609.32042/) (2026)
- [Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch](https://mlsanity.com/paper/2610.10592/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) play a central role in modern artificial intelligence, yet their development has been primarily focused on English, resulting in limited support for other languages. We present PLLuM (Polish Large Language Model), the largest open-source family of foundation models tailored specifically for the Polish language. Developed by a consortium of major Polish research institutions, PLLuM addresses the need for high-quality, transparent, and culturally relevant language models beyond the English-centric commercial landscape. We describe the development process, including the construction of a new 140-billion-token Polish text corpus for pre-training, a 77k custom instructions dataset, and a 100k preference optimization dataset. A key component is a Responsible AI framework that incorporates strict data governance and a hybrid module for output correction and safety filtering. We detail the models' architecture, training procedures, and alignment techniques for both base and instruction-tuned variants, and demonstrate their utility in a downstream task within public administration. By releasing these models publicly, PLLuM aims to foster open research and strengthen sovereign AI technologies in Poland.