MME-Industry: A Cross-Industry Multimodal Evaluation Benchmark
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
With the rapid advancement of Multimodal Large Language Models (MLLMs), numerous evaluation benchmarks have emerged. However, comprehensive assessments of their performance across diverse industrial applications remain limited. In this paper, we introduce MME-Industry, a novel benchmark designed specifically for evaluating MLLMs in industrial settings.The benchmark encompasses 21 distinct domain, comprising 1050 question-answer pairs with 50 questions per domain. To ensure data integrity and prevent potential leakage from public datasets, all question-answer pairs were manually crafted and validated by domain experts. Besides, the benchmark's complexity is effectively enhanced by incorporating non-OCR questions that can be answered directly, along with tasks requiring specialized domain knowledge. Moreover, we provide both Chinese and English versions of the benchmark, enabling comparative analysis of MLLMs' capabilities across these languages. Our findings contribute valuable insights into MLLMs' practical industrial applications and illuminate promising directions for future model optimization research.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 28 Jan 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models
- 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
- 2023MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
- 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models
- 2025MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams
- 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Models
- 2025Enterprise Large Language Model Evaluation Benchmark
- 2024A Survey on Benchmarks of Multimodal Large Language Models