mlsanityData to 7 Oct 2026

bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark

arXiv 2306.02349 ↗First posted 4 Jun 2023cs.CL cs.IR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We present bgGLUE(Bulgarian General Language Understanding Evaluation), a benchmark for evaluating language models on Natural Language Understanding (NLU) tasks in Bulgarian. Our benchmark includes NLU tasks targeting a variety of NLP problems (e.g., natural language inference, fact-checking, named entity recognition, sentiment analysis, question answering, etc.) and machine learning tasks (sequence labeling, document-level classification, and regression). We run the first systematic evaluation of pre-trained language models for Bulgarian, comparing and contrasting results across the nine tasks in the benchmark. The evaluation results show strong performance on sequence labeling tasks, but there is a lot of room for improvement for tasks that require more complex reasoning. We make bgGLUE publicly available together with the fine-tuning and the evaluation code, as well as a public leaderboard at https://bgglue.github.io/, and we hope that it will enable further advancements in developing NLU models for Bulgarian.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 69 of its 109 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

69 of 109 references.

References · 69

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  11. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  12. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  13. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  14. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  15. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  16. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  17. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  18. 2017Learning multiple visual domains with residual adapterscited by 553
  19. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  20. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  21. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  22. 2019CamemBERT: a Tasty French Language Modelcited by 368
  23. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  24. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  25. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  26. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  27. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  28. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  29. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  30. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  31. 2020German's Next Language Modelcited by 144
  32. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  33. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  34. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  35. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  36. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
  37. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  38. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025skLEP: A Slovak General Language Understanding Benchmarkcited by 3
  2. 2026ltzGLUE: Luxembourgish General Language Understanding Evaluationcited by 0
  3. 2026Sch"utzen: Evaluating LLM Safety in Bulgarian and German Contextscited by 0
  4. 2026Translation as a Computationally Efficient Bridge: Feasibility of English BERT for Low-Resource Languagescited by 0