TechImage-Bench: Rubric-Based Evaluation for Technical Image Generation
Not on the map. Why?
Abstract (arXiv)
We study technical image generation, where a model must synthesize information-dense, scientifically precise illustrations from detailed descriptions rather than merely produce visually plausible pictures. To quantify the progress, we introduce TechImage-Bench, a rubric-based benchmark that targets biology schematics, engineering/patent drawings, and general technical illustrations. For 654 figures collected from real textbooks and technical reports, we construct detailed image instructions and a hierarchy of rubrics that decompose correctness into 6,076 criteria and 44,131 binary checks. Rubrics are derived from surrounding text and reference figures using large multimodal models, and are evaluated by an automated LMM-based judge with a principled penalty scheme that aggregates sub-question outcomes into interpretable criterion scores. We benchmark several representative text-to-image models on TechImage-Bench and find that, despite strong open-domain performance, the best base model reaches only 0.801 rubric accuracy and 0.576 criterion score overall, revealing substantial gaps in fine-grained scientific fidelity. Finally, we show that the same rubrics provide actionable supervision: feeding failed checks back into an editing model for iterative refinement boosts a strong generator from 0.660 to 0.865 in rubric accuracy and from 0.382 to 0.697 in criterion score. TechImage-Bench thus offers both a rigorous diagnostic for technical image generation and a scalable signal for improving specification-faithful scientific illustrations.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 13 Dec 2025.
Builds on
38 of 51 references.
References · 38
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024GPT-4o System Cardcited by 4,588
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2016Improved Techniques for Training GANscited by 2,970
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
26 more
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 249
- 2020DeepSVG: A Hierarchical Generative Network for Vector Graphics Animationcited by 159
- 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
- 2023ARNIQA: Learning Distortion Manifold for Image Quality Assessmentcited by 89
- 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 79
- 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 76
- 2024CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusioncited by 72
- 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 66
- 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
- 2022OCR-VQGAN: Taming Text-within-Image Generationcited by 31
- 2024From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editingcited by 31
- 2025SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Modelcited by 19
- 2023FigGen: Text to Scientific Figure Generationcited by 17
- 2024SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinementcited by 15
- 2024Exploring Rich Subjective Quality Information for Image Quality Assessment in the Wildcited by 13
- 2024An Agentic Approach to Automatic Creation of P&ID Diagrams from Natural Language Descriptionscited by 12
- 2025MagicGeo: Training-Free Text-Guided Geometric Diagram Generationcited by 11
- 2024Exploring the Impact of AI-generated Image Tools on Professional and Non-professional Users in the Art and Design Fieldscited by 10
- 2025From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphicscited by 6
- 2017Creatism: A deep-learning photographer capable of creating professional workcited by 5
- 2024AI-Assisted Causal Pathway Diagram for Human-Centered Designcited by 4
- 2025InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generationcited by 4
- 2025TextPixs: Glyph-Conditioned Diffusion with Character-Aware Attention and OCR-Guided Supervisioncited by 2
- 2023Automated use case diagram generator using NLP and MLcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models
- 2026PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark
- 2026SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation
- 2026Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility
- 2026Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation
- 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrations
- 2026SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context
- 2026RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing