mlsanityData to 14 Oct 2026

Generating abstractive summaries of Lithuanian news articles using a transformer model

arXiv 2105.03279 ↗First posted 23 Apr 2021cs.CL cs.IR cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

In this work, we train the first monolingual Lithuanian transformer model on a relatively large corpus of Lithuanian news articles and compare various output decoding algorithms for abstractive news summarization. We achieve an average ROUGE-2 score 0.163, generated summaries are coherent and look impressive at first glance. However, some of them contain misleading information that is not so easy to spot. We describe all the technical details and share our trained model and accompanying code in an online open-source repository, as well as some characteristic samples of the generated summaries.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 23 Apr 2021.

Builds on

25 of 34 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  9. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  11. 2018Hierarchical Neural Story Generationcited by 1,193
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
13 more
  1. 2015Teaching Machines to Read and Comprehendcited by 1,049
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  3. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  6. 2013Generating Sequences With Recurrent Neural Networkscited by 554
  7. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  8. 2017Six Challenges for Neural Machine Translationcited by 335
  9. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  10. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  11. 2020GSum: A General Framework for Guided Neural Abstractive Summarizationcited by 129
  12. 2017Text Summarization Techniques: A Brief Surveycited by 59
  13. 2016Neural Machine Translation with Reconstructioncited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2022Towards Lithuanian grammatical error correctioncited by 2
  2. 2026Method for Aggregating Unstructured Data Using Large Language Modelscited by 0