Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition
Not on the map. Why?
Abstract (arXiv)
Visualization's design knowledge-effectiveness rankings, encoding guidelines, color models, preattentive processing rules -- derives from six decades of psychophysical studies of human vision. Yet vision-language models (VLMs) increasingly consume chart images in automated analysis pipelines, and a growing body of benchmark evidence indicates that this human-centered knowledge base does not straightforwardly transfer to machine audiences. Machines exhibit different encoding performance patterns, process images through patch-based tokenization rather than holistic perception, and fail on design patterns that pose no difficulty for humans-while occasionally succeeding where humans struggle. Current approaches address this gap primarily by bypassing vision entirely, converting charts to data tables or structured text. We argue that this response forecloses a more fundamental question: what visual representations would actually serve machine cognition well? This paper makes the case that the visualization field needs to investigate machine-oriented visual design as a distinct research problem. We synthesize evidence from VLM benchmarks, visual reasoning research, and visualization literacy studies to show that the human-machine perceptual divergence is qualitative, not merely quantitative, and critically examine the prevailing bypassing approach. We propose a conceptual distinction between human-oriented and machine-oriented visualization-not as an engineering architecture but as a recognition that different audiences may require fundamentally different design foundations-and outline a research agenda for developing the empirical foundations the field currently lacks: the beginnings of a "machine Bertin" to complement the human-centered knowledge the field already possesses.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 56 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
30 of 56 references.
References · 30
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2013Intriguing properties of neural networkscited by 5,155
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
- 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
- 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
- 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
- 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
- 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
- 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
- 2023LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Modelscited by 82
18 more
- 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Modelscited by 66
- 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answeringcited by 50
- 2021Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Contentcited by 39
- 2023Visual cognition in multimodal large language modelscited by 38
- 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Modelscited by 32
- 2024DataNarrative: Automated Data-Driven Storytelling with Visualizations and Textscited by 25
- 2024Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Waycited by 21
- 2023Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Modelscited by 20
- 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Testscited by 20
- 2022Rich Screen Reader Experiences for Accessible Data Visualizationcited by 12
- 2024DracoGPT: Extracting Visualization Design Preferences from Large Language Modelscited by 12
- 2024Universal dimensions of visual representationcited by 12
- 2025EncQA: Benchmarking Vision-Language Models on Visual Encodings for Chartscited by 12
- 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Modelscited by 7
- 2025CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Modelscited by 5
- 2025Losing the Plot: How VLM responses degrade on imperfect chartscited by 5
- 2025Does visualization help AI understand data?cited by 3
- 2025Vision-language models learn the geometry of human perceptual spacecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Do Vision-Language Models See Visualizations Like Humans? Alignment in Chart Categorization
- 2026Do MLLMs See What We See? Analyzing Visualization Literacy Barriers in AI Systems
- 2025Structured Visualization Design Knowledge for Grounding Generative Reasoning and Situated Feedback
- 2026How Do LLMs See Charts? A Comparative Study on High-Level Visualization Comprehension in Humans and LLMs ↗
- 2026Do Large Language Models Understand Data Visualization Principles?
- 2025Evaluating Graphical Perception with Multimodal LLMs
- 2024Understanding Graphical Perception in Data Visualization through Zero-shot Prompting of Vision-Language Models
- 2025The Plot Thickens: Quantitative Part-by-Part Exploration of MLLM Visualization Literacy