# Knesset-DictaBERT: A Hebrew Language Model for Parliamentary Proceedings

arXiv: https://arxiv.org/abs/2407.20581 · HTML: https://mlsanity.com/paper/2407.20581/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 4 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew](https://mlsanity.com/paper/2308.16687/) (2023)
- [The Knesset Corpus: An Annotated Corpus of Hebrew Parliamentary Proceedings](https://mlsanity.com/paper/2405.18115/) (2024)

## Built on by (most cited first, 1 of 1)

- [Unveiling Affective Polarization Trends in Parliamentary Proceedings](https://mlsanity.com/paper/2512.05231/) (2025)

## Abstract (arXiv)

We present Knesset-DictaBERT, a large Hebrew language model fine-tuned on the Knesset Corpus, which comprises Israeli parliamentary proceedings. The model is based on the DictaBERT architecture and demonstrates significant improvements in understanding parliamentary language according to the MLM task. We provide a detailed evaluation of the model's performance, showing improvements in perplexity and accuracy over the baseline DictaBERT model.