# Algorithmic Ways of Seeing: Using Object Detection to Facilitate Art Exploration

arXiv: https://arxiv.org/abs/2403.19174 · HTML: https://mlsanity.com/paper/2403.19174/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 94 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions](https://mlsanity.com/paper/1705.02498/) (2017)
- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [The Creativity of Text-to-Image Generation](https://mlsanity.com/paper/2206.02904/) (2022)
- [Understanding and Creating Art with AI: Review and Outlook](https://mlsanity.com/paper/2102.09109/) (2021)
- [Detecting People in Artwork with CNNs](https://mlsanity.com/paper/1610.08871/) (2016)
- [Improving Object Detection in Art Images Using Only Style Transfer](https://mlsanity.com/paper/2102.06529/) (2021)
- [Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry](https://mlsanity.com/paper/2101.04035/) (2021)
- [Semi-supervised Human Pose Estimation in Art-historical Images](https://mlsanity.com/paper/2207.02976/) (2022)
- [Artistic Domain Generalisation Methods are Limited by their Deep Representations](https://mlsanity.com/paper/1907.12622/) (2019)
- [Sensitive Pictures: Emotional Interpretation in the Museum](https://mlsanity.com/paper/2203.01041/) (2022)
- [Designing for Interpersonal Museum Experiences](https://mlsanity.com/paper/2011.11398/) (2020)
- [Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities](https://mlsanity.com/paper/2208.13512/) (2022)

## Built on by (most cited first, 4 of 4)

- [Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation](https://mlsanity.com/paper/2409.14634/) (2024)
- [No Annotations for Object Detection in Art through Stable Diffusion](https://mlsanity.com/paper/2412.06286/) (2024)
- [KidsArtBench: Multi-Dimensional Children's Art Evaluation with Attribute-Aware MLLMs](https://mlsanity.com/paper/2512.12503/) (2025)
- [If These Walls Could Talk: Critical Play with Large Language Models in Museums](https://mlsanity.com/paper/2606.15565/) (2026)

## Abstract (arXiv)

This Research through Design paper explores how object detection may be applied to a large digital art museum collection to facilitate new ways of encountering and experiencing art. We present the design and evaluation of an interactive application called SMKExplore, which allows users to explore a museum's digital collection of paintings by browsing through objects detected in the images, as a novel form of open-ended exploration. We provide three contributions. First, we show how an object detection pipeline can be integrated into a design process for visual exploration. Second, we present the design and development of an app that enables exploration of an art museum's collection. Third, we offer reflections on future possibilities for museums and HCI researchers to incorporate object detection techniques into the digitalization of museums.