Large Language Models for Summarizing Czech Historical Documents and Beyond
Not on the map. Why?
Abstract (arXiv)
Text summarization is the task of shortening a larger body of text into a concise version while retaining its essential meaning and key information. While summarization has been significantly explored in English and other high-resource languages, Czech text summarization, particularly for historical documents, remains underexplored due to linguistic complexities and a scarcity of annotated datasets. Large language models such as Mistral and mT5 have demonstrated excellent results on many natural language processing tasks and languages. Therefore, we employ these models for Czech summarization, resulting in two key contributions: (1) achieving new state-of-the-art results on the modern Czech summarization dataset SumeCzech using these advanced models, and (2) introducing a novel dataset called Posel od vCerchova for summarization of historical Czech documents with baseline results. Together, these contributions provide a great potential for advancing Czech text summarization and open new avenues for research in Czech historical text processing.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 14 Aug 2025.
Builds on
19 of 34 references.
References · 19
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2015Teaching Machines to Read and Comprehendcited by 1,049
7 more
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
- 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
- 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
- 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
- 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
- 2018Tensor2Tensor for Neural Machine Translationcited by 85
- 2019Leveraging BERT for Extractive Text Summarization on Lecturescited by 63
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.