mlsanityData to 14 Oct 2026

Bielik 11B v3: Multilingual Large Language Model for European Languages

arXiv 2601.11579 ↗First posted 30 Dec 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

We present Bielik 11B v3, a state-of-the-art language model highly optimized for the Polish language, while also maintaining strong capabilities in other European languages. This model extends the Mistral 7B v0.2 architecture, scaled to 11B parameters via depth up-scaling. Its development involved a comprehensive four-stage training pipeline: continuous pre-training, supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and reinforcement learning. Comprehensive evaluations demonstrate that Bielik 11B v3 achieves exceptional performance. It significantly surpasses other specialized Polish language models and outperforms many larger models (with 2-6 times more parameters) on a wide range of tasks, from basic linguistic understanding to complex reasoning. The model's parameter efficiency, combined with extensive quantization options, allows for effective deployment across diverse hardware configurations. Bielik 11B v3 not only advances AI capabilities for the Polish language but also establishes a new benchmark for developing resource-efficient, high-performance models for less-represented languages.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 30 Dec 2025.

Builds on

28 of 42 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2020GLU Variants Improve Transformercited by 1,804
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  8. 2016Language Modeling with Gated Convolutional Networkscited by 702
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  10. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  11. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  12. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
16 more
  1. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  2. 2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Languagecited by 143
  3. 2024EuroLLM: Multilingual Language Models for Europecited by 106
  4. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 86
  5. 2023EQ-Bench: An Emotional Intelligence Benchmark for Large Language Modelscited by 80
  6. 2020KLEJ: Comprehensive Benchmark for Polish Language Understandingcited by 51
  7. 2024Instruction Tuning With Loss Over Instructionscited by 40
  8. 2023Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformerscited by 30
  9. 2019Evaluation of Sentence Representations in Polishcited by 18
  10. 2024Bielik 7B v0.1: A Polish Language Model -- Development, Insights, and Evaluationcited by 15
  11. 2022Training Effective Neural Sentence Encoders from Automatically Mined Paraphrasescited by 11
  12. 2024Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuningcited by 11
  13. 2024GPT-4 passes most of the 297 written Polish Board Certification Examinationscited by 10
  14. 2022PolQA: Polish Question Answering Datasetcited by 7
  15. 2025Evaluating Polish linguistic and cultural competency in large language modelscited by 6
  16. 2025Bielik v3 Small: Technical Reportcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Bielik Guard: Efficient Polish Language Safety Classifiers for LLM Content Moderationcited by 2
  2. 2026Advancing Polish Language Modeling through Tokenizer Optimization in the Bielik v3 7B and 11B Seriescited by 1
  3. 2026LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metricscited by 0
  4. 2026Retrieval-Augmented Large Language Models as Components of Cognitive Computing architecture for Regulatory Knowledge Managementcited by 0
  5. 2026Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bitcited by 0
  6. 2026Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Bielik 11B v2 Technical Report
  2. 2026Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language
  3. 2025PLLuM: A Family of Polish Large Language Models
  4. 2026Making Bielik LLM Reason (Better): A Field Report
  5. 2026Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model
  6. 2024Multilingual Large Language Models: A Systematic Survey
  7. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
  8. 2024Lens: Rethinking Multilingual Enhancement for Large Language Models