# Alvorada-Bench: Can Language Models Solve Brazilian University Entrance Exams?

arXiv: https://arxiv.org/abs/2508.15835 · HTML: https://mlsanity.com/paper/2508.15835/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 6 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 5 of 5)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs](https://mlsanity.com/paper/2410.15956/) (2024)
- [BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams](https://mlsanity.com/paper/2307.05410/) (2023)
- [Comparing large language models and human programmers for generating programming code](https://mlsanity.com/paper/2403.00894/) (2024)
- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)

## Built on by (most cited first, 1 of 1)

- [Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese](https://mlsanity.com/paper/2606.07853/) (2026)

## Abstract (arXiv)

Language models are increasingly used in Brazil, but most evaluation remains English-centric. This paper presents Alvorada-Bench, a 4,515-question, text-only benchmark drawn from five Brazilian university entrance examinations. Evaluating twenty models under zero-shot, role-playing, and chain-of-thought prompting, producing 270,900 responses with structured self-reports of confidence, perceived difficulty, and Bloom level. The top models exceed 94% accuracy overall, but accuracy declines on Mathematics and on the engineering oriented IME and ITA exams, indicating persistent weaknesses in multi-step reasoning. Confidence is well calibrated and correlates with perceived difficulty, revealing that models can accurately assess their own certainty capabilities. A cost accuracy analysis shows that high accuracy is achievable at under 2 per 1K tokens. On ENEM 2024 the top model (O3) achieved perfect scores in Languages subject questions while even the weakest system (GPT-4.1 Nano) only underperforms humans in Mathematics. Through exams that distill decades of Brazilian educational priorities and assess millions of students yearly, Alvorada-Bench establishes whether language models can navigate the intersection of language, culture, and reasoning that defines academic readiness in Brazil.