mlsanityData to 14 Oct 2026

TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation

arXiv 2401.14373 ↗First posted 25 Jan 2024cs.CL cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

The recent advances in natural language processing have predominantly favored well-resourced English-centric models, resulting in a significant gap with low-resource languages. In this work, we introduce the language model TURNA, which is developed for the low-resource language Turkish and is capable of both natural language understanding and generation tasks. TURNA is pretrained with an encoder-decoder architecture based on the unified framework UL2 with a diverse corpus that we specifically curated for this purpose. We evaluated TURNA with three generation tasks and five understanding tasks for Turkish. The results show that TURNA outperforms several multilingual models in both understanding and generation tasks, and competes with monolingual Turkish models in understanding tasks. TURNA is made available at https://huggingface.co/boun-tabi-LMG/TURNA .

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 33 of its 66 references that are in the map. First posted on arXiv 25 Jan 2024.

Builds on

33 of 66 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  10. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
21 more
  1. 2020GLU Variants Improve Transformercited by 1,804
  2. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  4. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  5. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  6. 2022Galactica: A Large Language Model for Sciencecited by 744
  7. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  8. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  9. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  10. 2022UL2: Unifying Language Learning Paradigmscited by 295
  11. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  12. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
  13. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  14. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  15. 2020MLSUM: The Multilingual Summarization Corpuscited by 121
  16. 2019Multilingual is not enough: BERT for Finnishcited by 119
  17. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 106
  18. 2020ConvBERT: Improving BERT with Span-based Dynamic Convolutioncited by 88
  19. 2020What the [MASK]? Making Sense of Language-Specific BERT Modelscited by 44
  20. 2023An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Textcited by 23
  21. 2020EstBERT: A Pretrained Language-Specific BERT for Estoniancited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarkingcited by 10
  3. 2024Introducing cosmosGPT: Monolingual Training for Turkish Language Modelscited by 8
  4. 2025Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkishcited by 8
  5. 2024LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Languagecited by 6
  6. 2025TiSpell: A Semi-Masked Methodology for Tibetan Spelling Correction covering Multi-Level Error with Data Augmentationcited by 2
  7. 2025TabiBERT: A Large-Scale ModernBERT Foundation Model and A Unified Benchmark for Turkishcited by 1
  8. 2026TurkicNLP: An NLP Toolkit for Turkic Languagescited by 1
  9. 2024T"urkcce Dil Modellerinin Performans Karcsilacstirmasi Performance Comparison of Turkish Language Modelscited by 0
  10. 2025Building Foundations for Natural Language Processing of Historical Turkish: Resources and Modelscited by 0
  11. 2026Domain Specific Specialization in Low-Resource Settings: The Efficacy of Offline Response-Based Knowledge Distillation in Large Language Modelscited by 0
  12. 2026Diffutron: A Masked Diffusion Language Model for Turkish Languagecited by 0
1 more
  1. 2026TurEngMix: A Text Corpus and Benchmark for Turkish-English Code-Mixed Language Identification and Named Entity Recognitioncited by 0