mlsanityData to 14 Oct 2026

MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessment

arXiv 2508.17290 ↗First posted 24 Aug 2025cs.AI cs.LGcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent advancements in large vision-language models (VLMs) have primarily focused on English, with limited attention given to other languages. To address this gap, we introduce MEENA (also known as PersianMMMU), the first dataset designed to evaluate Persian VLMs across scientific, reasoning, and human-level understanding tasks. Our dataset comprises approximately 7,500 Persian and 3,000 English questions, covering a wide range of topics such as reasoning, mathematics, physics, diagrams, charts, and Persian art and literature. Key features of MEENA include: (1) diverse subject coverage spanning various educational levels, from primary to upper secondary school, (2) rich metadata, including difficulty levels and descriptive answers, (3) original Persian data that preserves cultural nuances, (4) a bilingual structure to assess cross-linguistic performance, and (5) a series of diverse experiments assessing various capabilities, including overall performance, the model's ability to attend to images, and its tendency to generate hallucinations. We hope this benchmark contributes to enhancing VLM capabilities beyond English.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 60 references that are in the map. First posted on arXiv 24 Aug 2025.

Builds on

38 of 60 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Vision-Language Models Can't See the Obvious7 Jul
  2. 2025Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding17 Feb
  3. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination15 Nov
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2024A Survey on LLM-as-a-Judgecited by 1,423
  10. 2016A Diagram Is Worth A Dozen Imagescited by 804
  11. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  12. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
22 more
  1. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  2. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  3. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  4. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 256
  5. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  6. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 210
  7. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  8. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 203
  9. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  10. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
  11. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  12. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
  13. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  14. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoningcited by 70
  15. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 69
  16. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  17. 2024MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Taskscited by 41
  18. 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
  19. 2024Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?cited by 22
  20. 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuningcited by 11
  21. 2025EAGLE: Enhanced Visual Grounding Minimizes Hallucinations in Instructional Multimodal Modelscited by 8
  22. 2025Generative AI for Character Animation: A Comprehensive Survey of Techniques, Applications, and Future Directionscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoningcited by 1
  2. 2026Unmasking the Factual-Conceptual Gap in Persian Language Modelscited by 1
  3. 2026Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models
  2. 2024PALO: A Polyglot Large Multimodal Model for 5B People
  3. 2025PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Models
  4. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models
  5. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
  6. 2024Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT
  7. 2025Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
  8. 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models