# Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition arXiv: https://arxiv.org/abs/2602.01527 ยท HTML: https://mlsanity.com/paper/2602.01527/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Builds on 30 of its 56 references that are in the map. - First posted on arXiv 2 Feb 2026. ## Builds on (most cited first, 30 of 30) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013) - [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022) - [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018) - [Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models](https://mlsanity.com/paper/2406.09403/) (2024) - [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024) - [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023) - [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022) - [UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2305.14761/) (2023) - [MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering](https://mlsanity.com/paper/2212.09662/) (2022) - [Towards Interpreting Visual Information Processing in Vision-Language Models](https://mlsanity.com/paper/2410.07149/) (2024) - [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023) - [Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2404.03622/) (2024) - [ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering](https://mlsanity.com/paper/2405.07001/) (2024) - [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021) - [Visual cognition in multimodal large language models](https://mlsanity.com/paper/2311.16093/) (2023) - [ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models](https://mlsanity.com/paper/2505.13444/) (2025) - [DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts](https://mlsanity.com/paper/2408.05346/) (2024) - [Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way](https://mlsanity.com/paper/2408.16119/) (2024) - [Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Models](https://mlsanity.com/paper/2309.10245/) (2023) - [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025) - [Rich Screen Reader Experiences for Accessible Data Visualization](https://mlsanity.com/paper/2205.04917/) (2022) - [DracoGPT: Extracting Visualization Design Preferences from Large Language Models](https://mlsanity.com/paper/2408.06845/) (2024) - [Universal dimensions of visual representation](https://mlsanity.com/paper/2408.12804/) (2024) - [EncQA: Benchmarking Vision-Language Models on Visual Encodings for Charts](https://mlsanity.com/paper/2508.04650/) (2025) - [The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models](https://mlsanity.com/paper/2508.09716/) (2025) - [CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models](https://mlsanity.com/paper/2505.17202/) (2025) - [Losing the Plot: How VLM responses degrade on imperfect charts](https://mlsanity.com/paper/2509.18425/) (2025) - [Does visualization help AI understand data?](https://mlsanity.com/paper/2507.18022/) (2025) - [Vision-language models learn the geometry of human perceptual space](https://mlsanity.com/paper/2510.20859/) (2025) ## Abstract (arXiv) Visualization's design knowledge-effectiveness rankings, encoding guidelines, color models, preattentive processing rules -- derives from six decades of psychophysical studies of human vision. Yet vision-language models (VLMs) increasingly consume chart images in automated analysis pipelines, and a growing body of benchmark evidence indicates that this human-centered knowledge base does not straightforwardly transfer to machine audiences. Machines exhibit different encoding performance patterns, process images through patch-based tokenization rather than holistic perception, and fail on design patterns that pose no difficulty for humans-while occasionally succeeding where humans struggle. Current approaches address this gap primarily by bypassing vision entirely, converting charts to data tables or structured text. We argue that this response forecloses a more fundamental question: what visual representations would actually serve machine cognition well? This paper makes the case that the visualization field needs to investigate machine-oriented visual design as a distinct research problem. We synthesize evidence from VLM benchmarks, visual reasoning research, and visualization literacy studies to show that the human-machine perceptual divergence is qualitative, not merely quantitative, and critically examine the prevailing bypassing approach. We propose a conceptual distinction between human-oriented and machine-oriented visualization-not as an engineering architecture but as a recognition that different audiences may require fundamentally different design foundations-and outline a research agenda for developing the empirical foundations the field currently lacks: the beginnings of a "machine Bertin" to complement the human-centered knowledge the field already possesses.