Culturally Grounded Physical Commonsense Reasoning in Italian and English: A Submission to the MRL 2025 Shared Task
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
This paper presents our submission to the MRL 2025 Shared Task on Multilingual Physical Reasoning Datasets. The objective of the shared task is to create manually-annotated evaluation data in the physical commonsense reasoning domain, for languages other than English, following a format similar to PIQA. Our contribution, FormaMentis, is a novel benchmark for physical commonsense reasoning that is grounded in Italian language and culture. The data samples in FormaMentis are created by expert annotators who are native Italian speakers and are familiar with local customs and norms. The samples are additionally translated into English, while preserving the cultural elements unique to the Italian context.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 12 references that are in the map. First posted on arXiv 26 Oct 2025.
Builds on
9 of 12 references.
Closest ancestors · 3
Same direction, newest first
- 2025HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning17 Feb
- 2025A Survey of Multilingual Reasoning in Language Models13 Feb
- 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct
Other references · 6
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
- 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
- 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
- 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 65
- 2019When Choosing Plausible Alternatives, Clever Hans can be Clevercited by 40
- 2022"John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibilitycited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: a Study on Basque
- 2025Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
- 2025Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context
- 2025Everyday Physics in Korean Contexts: A Culturally Grounded Physical Reasoning Benchmark
- 2024Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning
- 2026ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluation
- 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans
- 2023Probing Physical Reasoning with Counter-Commonsense Context