# Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams

arXiv: https://arxiv.org/abs/2303.17003 · HTML: https://mlsanity.com/paper/2303.17003/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [GPT Takes the Bar Exam](https://mlsanity.com/paper/2212.14402/) (2022)
- [GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities](https://mlsanity.com/paper/2301.04408/) (2023)

## Built on by (most cited first, 25 of 25)

- [LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities](https://mlsanity.com/paper/2305.13168/) (2023)
- [Neural Retrievers are Biased Towards LLM-Generated Content](https://mlsanity.com/paper/2310.20501/) (2023)
- [Sabi'a: Portuguese Large Language Models](https://mlsanity.com/paper/2304.07880/) (2023)
- [Advancing Generative AI for Portuguese with Open Decoder Gerv'asio PT*](https://mlsanity.com/paper/2402.18766/) (2024)
- [BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams](https://mlsanity.com/paper/2307.05410/) (2023)
- [GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models](https://mlsanity.com/paper/2310.06225/) (2023)
- [Sabi'a-2: A New Generation of Portuguese Large Language Models](https://mlsanity.com/paper/2403.09887/) (2024)
- [A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course](https://mlsanity.com/paper/2403.16977/) (2024)
- [Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corpora](https://mlsanity.com/paper/2509.08824/) (2025)
- [Evaluating GPT-4's Vision Capabilities on Brazilian University Admission Exams](https://mlsanity.com/paper/2311.14169/) (2023)
- [Evaluating GPT-3.5 and GPT-4 on Grammatical Error Correction for Brazilian Portuguese](https://mlsanity.com/paper/2306.15788/) (2023)
- [Juru: Legal Brazilian Large Language Model from Reputable Sources](https://mlsanity.com/paper/2403.18140/) (2024)
- [PoETa v2: Toward More Robust Evaluation of Large Language Models in Portuguese](https://mlsanity.com/paper/2511.17808/) (2025)
- [Professional Certification Benchmark Dataset: The First 500 Jobs For Large Language Models](https://mlsanity.com/paper/2305.05377/) (2023)
- [Examining the Behavior of LLM Architectures Within the Framework of Standardized National Exams in Brazil](https://mlsanity.com/paper/2408.05035/) (2024)
- [Assessing the Capability of LLMs in Solving POSCOMP Questions](https://mlsanity.com/paper/2505.20338/) (2025)
- [Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners](https://mlsanity.com/paper/2408.05204/) (2024)
- [Benchmarking quantized LLaMa-based models on the Brazilian Secondary School Exam](https://mlsanity.com/paper/2309.12071/) (2023)
- [Evaluating AI Vocational Skills Through Professional Testing](https://mlsanity.com/paper/2312.10603/) (2023)
- ['Evaluation des capacit'es de r'eponse de larges mod`eles de langage (LLM) pour des questions d'historiens](https://mlsanity.com/paper/2406.15173/) (2024)
- [Impact of ChatGPT on the writing style of condensed matter physicists](https://mlsanity.com/paper/2408.17325/) (2024)
- [Prosa: Rubric-Based Evaluation of LLMs on Real User Chats in Brazilian Portuguese](https://mlsanity.com/paper/2605.01630/) (2026)
- [Toten: A Knowledge-Based System For Structure-Preserving Representation Of Physical Quantities And Technical Notation In Brazilian Portuguese](https://mlsanity.com/paper/2606.19626/) (2026)
- [BLUEX v2: Benchmarking LLMs on Open-Ended Questions from Brazilian University Entrance Exams](https://mlsanity.com/paper/2606.22723/) (2026)
- [MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages](https://mlsanity.com/paper/2607.00890/) (2026)

## Abstract (arXiv)

The present study aims to explore the capabilities of Language Models (LMs) in tackling high-stakes multiple-choice tests, represented here by the Exame Nacional do Ensino M'edio (ENEM), a multidisciplinary entrance examination widely adopted by Brazilian universities. This exam poses challenging tasks for LMs, since its questions may span into multiple fields of knowledge, requiring understanding of information from diverse domains. For instance, a question may require comprehension of both statistics and biology to be solved. This work analyzed responses generated by GPT-3.5 and GPT-4 models for questions presented in the 2009-2017 exams, as well as for questions of the 2022 exam, which were made public after the training of the models was completed. Furthermore, different prompt strategies were tested, including the use of Chain-of-Thought (CoT) prompts to generate explanations for answers. On the 2022 edition, the best-performing model, GPT-4 with CoT, achieved an accuracy of 87%, largely surpassing GPT-3.5 by 11 points. The code and data used on experiments are available at https://github.com/piresramon/gpt-4-enem.