mlsanityData to 14 Oct 2026

IndustryBench-MIPU: Benchmarking Multi-Image Attribute Value Extraction for Industrial Products

arXiv 2606.14383 ↗First posted 12 Jun 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Industrial products such as valves and circuit breakers are defined by dense technical specifications that govern procurement, compatibility, and safety across supply chains. These specifications are scattered across multiple heterogeneous product images, including specification tables, nameplates, and technical drawings, yet whether Multimodal Large Language Models (MLLMs) can reliably recover them remains underexplored. To fill this gap, we introduce IndustryBench-MIPU, the first large-scale benchmark for multi-image industrial product understanding, built around structured attribute extraction -- recovering property-value pairs from product images. This task jointly probes text recognition on specification tables and nameplates, visual reasoning over technical drawings, domain knowledge to decode industrial terminology, and cross-image evidence integration to assemble scattered specifications. Concretely, the benchmark comprises 4,559 products across 27,652 images with 103,703 annotations spanning 18 industrial categories, constructed through multi-model consensus and three-tier quality assurance. Evaluating nine MLLMs under both single-image and product-level multi-image settings reveals a stark completeness gap: models achieve high precision (86--94%) but the best recovers only 49.9% of product-level attributes; moving from single-image to multi-image extraction costs 15--34 percentage points of recall. Multi-image completeness, not single-image accuracy, is the core bottleneck. Dataset and code are publicly available.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

18 of 25 references.

References · 18

  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2019Towards VQA Models That Can Readcited by 1,718
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  6. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  7. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  8. 2018OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]cited by 63
  9. 2021MAVE: A Product Dataset for Multi-source Attribute Value Extractioncited by 53
  10. 2020Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Productcited by 39
  11. 2025Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmarkcited by 21
  12. 2024ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extractioncited by 17
6 more
  1. 2024EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLMcited by 12
  2. 2025EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commercecited by 8
  3. 2026IndustryBench: Probing the Industrial Knowledge Boundaries of LLMscited by 4
  4. 2026FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarioscited by 2
  5. 2025VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Modelscited by 1
  6. 2026VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Grounded Scaling: Why Agentic AI Needs Deterministic Environmentscited by 0
  2. 2026CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoningcited by 0
  3. 2026IndustryLLM: Failure-Driven LLM Training for Industrial Procurementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
  2. 2025Retrieval-Augmented Multi-LLM Ensemble for Industrial Part Specification Extraction
  3. 2025Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset
  4. 2025From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledge
  5. 2025Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?
  6. 2025A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model
  7. 2024MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection
  8. 2026Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams