mlsanityData to 7 Oct 2026

Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams

arXiv 2303.17003 ↗First posted 29 Mar 2023cs.CL cs.AI cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

The present study aims to explore the capabilities of Language Models (LMs) in tackling high-stakes multiple-choice tests, represented here by the Exame Nacional do Ensino M'edio (ENEM), a multidisciplinary entrance examination widely adopted by Brazilian universities. This exam poses challenging tasks for LMs, since its questions may span into multiple fields of knowledge, requiring understanding of information from diverse domains. For instance, a question may require comprehension of both statistics and biology to be solved. This work analyzed responses generated by GPT-3.5 and GPT-4 models for questions presented in the 2009-2017 exams, as well as for questions of the 2022 exam, which were made public after the training of the models was completed. Furthermore, different prompt strategies were tested, including the use of Chain-of-Thought (CoT) prompts to generate explanations for answers. On the 2022 edition, the best-performing model, GPT-4 with CoT, achieved an accuracy of 87%, largely surpassing GPT-3.5 by 11 points. The code and data used on experiments are available at https://github.com/piresramon/gpt-4-enem.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 17 of its 23 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

17 of 23 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  11. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
5 more
  1. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  2. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  3. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  4. 2022GPT Takes the Bar Examcited by 65
  5. 2023GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilitiescited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 8; 2025: 3; 2026: 4.

25 papers.

Sort:

By area:

  1. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  2. 2023Neural Retrievers are Biased Towards LLM-Generated Contentcited by 37
  3. 2023Sabi'a: Portuguese Large Language Modelscited by 19
  4. 2024Advancing Generative AI for Portuguese with Open Decoder Gerv'asio PT*cited by 19
  5. 2023BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXamscited by 15
  6. 2023GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Modelscited by 12
  7. 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
  8. 2024A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Coursecited by 10
  9. 2025Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corporacited by 7
  10. 2023Evaluating GPT-4's Vision Capabilities on Brazilian University Admission Examscited by 6
  11. 2023Evaluating GPT-3.5 and GPT-4 on Grammatical Error Correction for Brazilian Portuguesecited by 5
  12. 2024Juru: Legal Brazilian Large Language Model from Reputable Sourcescited by 5
13 more
  1. 2025PoETa v2: Toward More Robust Evaluation of Large Language Models in Portuguesecited by 5
  2. 2023Professional Certification Benchmark Dataset: The First 500 Jobs For Large Language Modelscited by 2
  3. 2024Examining the Behavior of LLM Architectures Within the Framework of Standardized National Exams in Brazilcited by 2
  4. 2025Assessing the Capability of LLMs in Solving POSCOMP Questionscited by 2
  5. 2024Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learnerscited by 1
  6. 2023Benchmarking quantized LLaMa-based models on the Brazilian Secondary School Examcited by 0
  7. 2023Evaluating AI Vocational Skills Through Professional Testingcited by 0
  8. 2024'Evaluation des capacit'es de r'eponse de larges mod`eles de langage (LLM) pour des questions d'historienscited by 0
  9. 2024Impact of ChatGPT on the writing style of condensed matter physicistscited by 0
  10. 2026Prosa: Rubric-Based Evaluation of LLMs on Real User Chats in Brazilian Portuguesecited by 0
  11. 2026Toten: A Knowledge-Based System For Structure-Preserving Representation Of Physical Quantities And Technical Notation In Brazilian Portuguesecited by 0
  12. 2026BLUEX v2: Benchmarking LLMs on Open-Ended Questions from Brazilian University Entrance Examscited by 0
  13. 2026MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Alvorada-Bench: Can Language Models Solve Brazilian University Entrance Exams?
  2. 2024Evaluating ChatGPT-4 Vision on Brazil's National Undergraduate Computer Science Exam
  3. 2025Performance of leading large language models in May 2025 in Membership of the Royal College of General Practitioners-style examination questions: a cross-sectional analysis
  4. 2024Evaluating Students' Open-ended Written Responses with LLMs: Using the RAG Framework for GPT-3.5, GPT-4, Claude-3, and Mistral-Large
  5. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models
  6. 2024Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency
  7. 2022Can large language models reason about medical questions?
  8. 2025Evaluating Large Language Models on the 2026 Korean CSAT Mathematics Exam: Measuring Mathematical Ability in a Zero-Data-Leakage Setting