CLARIN-PT-LDB: An Open LLM Leaderboard for Portuguese to assess Language, Culture and Civility
Not on the map. Why?
Abstract (arXiv)
This paper reports on the development of a leaderboard of Open Large Language Models (LLM) for European Portuguese (PT-PT), and on its associated benchmarks. This leaderboard comes as a way to address a gap in the evaluation of LLM for European Portuguese, which so far had no leaderboard dedicated to this variant of the language. The paper also reports on novel benchmarks, including some that address aspects of performance that so far have not been available in benchmarks for European Portuguese, namely model safeguards and alignment to Portuguese culture. The leaderboard is available at https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
15 of 29 references.
References · 15
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
- 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 195
- 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
- 2024Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 48
3 more
- 2024Advancing Generative AI for Portuguese with Open Decoder Gerv'asio PT*cited by 19
- 2024Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russiancited by 12
- 2024PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguesecited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.