# IndustryBench-MIPU: Benchmarking Multi-Image Attribute Value Extraction for Industrial Products

arXiv: https://arxiv.org/abs/2606.14383 · HTML: https://mlsanity.com/paper/2606.14383/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023)
- [MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models](https://mlsanity.com/paper/2408.02718/) (2024)
- [OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]](https://mlsanity.com/paper/1806.01264/) (2018)
- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product](https://mlsanity.com/paper/2009.07162/) (2020)
- [Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark](https://mlsanity.com/paper/2506.04280/) (2025)
- [ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction](https://mlsanity.com/paper/2404.15592/) (2024)
- [EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM](https://mlsanity.com/paper/2404.08886/) (2024)
- [EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce](https://mlsanity.com/paper/2512.08868/) (2025)
- [IndustryBench: Probing the Industrial Knowledge Boundaries of LLMs](https://mlsanity.com/paper/2605.10267/) (2026)
- [FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios](https://mlsanity.com/paper/2604.07413/) (2026)
- [VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models](https://mlsanity.com/paper/2508.11801/) (2025)
- [VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation](https://mlsanity.com/paper/2605.02035/) (2026)

## Built on by (most cited first, 3 of 3)

- [Grounded Scaling: Why Agentic AI Needs Deterministic Environments](https://mlsanity.com/paper/2606.22495/) (2026)
- [CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning](https://mlsanity.com/paper/2609.03526/) (2026)
- [IndustryLLM: Failure-Driven LLM Training for Industrial Procurement](https://mlsanity.com/paper/2609.31871/) (2026)

## Abstract (arXiv)

Industrial products such as valves and circuit breakers are defined by dense technical specifications that govern procurement, compatibility, and safety across supply chains. These specifications are scattered across multiple heterogeneous product images, including specification tables, nameplates, and technical drawings, yet whether Multimodal Large Language Models (MLLMs) can reliably recover them remains underexplored. To fill this gap, we introduce IndustryBench-MIPU, the first large-scale benchmark for multi-image industrial product understanding, built around structured attribute extraction -- recovering property-value pairs from product images. This task jointly probes text recognition on specification tables and nameplates, visual reasoning over technical drawings, domain knowledge to decode industrial terminology, and cross-image evidence integration to assemble scattered specifications. Concretely, the benchmark comprises 4,559 products across 27,652 images with 103,703 annotations spanning 18 industrial categories, constructed through multi-model consensus and three-tier quality assurance. Evaluating nine MLLMs under both single-image and product-level multi-image settings reveals a stark completeness gap: models achieve high precision (86--94%) but the best recovers only 49.9% of product-level attributes; moving from single-image to multi-image extraction costs 15--34 percentage points of recall. Multi-image completeness, not single-image accuracy, is the core bottleneck. Dataset and code are publicly available.