# RoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Models

arXiv: https://arxiv.org/abs/2509.25813 · HTML: https://mlsanity.com/paper/2509.25813/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?](https://mlsanity.com/paper/2402.12483/) (2024)
- [Model Equality Testing: Which Model Is This API Serving?](https://mlsanity.com/paper/2410.20247/) (2024)
- [DocTr: Document Transformer for Structured Information Extraction in Documents](https://mlsanity.com/paper/2307.07929/) (2023)
- [RoQLlama: A Lightweight Romanian Adapted Language Model](https://mlsanity.com/paper/2410.04269/) (2024)

## Abstract (arXiv)

In recent years, large language models (LLMs) have demonstrated significant potential across various natural language processing (NLP) tasks. However, their performance in domain-specific applications and non-English languages remains less explored. This study introduces a novel Romanian-language dataset for multiple-choice biology questions, carefully curated to assess LLM comprehension and reasoning capabilities in scientific contexts. Containing approximately 14,000 questions, the dataset provides a comprehensive resource for evaluating and improving LLM performance in biology. We benchmark several popular LLMs, analyzing their accuracy, reasoning patterns, and ability to understand domain-specific terminology and linguistic nuances. Additionally, we perform comprehensive experiments to evaluate the impact of prompt engineering, fine-tuning, and other optimization techniques on model performance. Our findings highlight both the strengths and limitations of current LLMs in handling specialized knowledge tasks in low-resource languages, offering valuable insights for future research and development.