# DeepVA: Bridging Cognition and Computation through Semantic Interaction and Deep Learning

arXiv: https://arxiv.org/abs/2007.15800 · HTML: https://mlsanity.com/paper/2007.15800/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 4 of 4)

- [Mixed Multi-Model Semantic Interaction for Graph-based Narrative Visualizations](https://mlsanity.com/paper/2302.06452/) (2023)
- [DeepSI: Interactive Deep Learning for Semantic Interaction](https://mlsanity.com/paper/2305.18357/) (2023)
- [LLM-Augmented Semantic Steering of Text Embedding Projection Spaces](https://mlsanity.com/paper/2605.01957/) (2026)
- [KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback](https://mlsanity.com/paper/2607.20556/) (2026)

## Abstract (arXiv)

This paper examines how deep learning (DL) representations, in contrast to traditional engineered features, can support semantic interaction (SI) in visual analytics. SI attempts to model user's cognitive reasoning via their interaction with data items, based on the data features. We hypothesize that DL representations contain meaningful high-level abstractions that can better capture users' high-level cognitive intent. To bridge the gap between cognition and computation in visual analytics, we propose DeepVA (Deep Visual Analytics), which uses high-level deep learning representations for semantic interaction instead of low-level hand-crafted data features. To evaluate DeepVA and compare to SI models with lower-level features, we design and implement a system that extends a traditional SI pipeline with features at three different levels of abstraction. To test the relationship between task abstraction and feature abstraction in SI, we perform visual concept learning tasks at three different task abstraction levels, using semantic interaction with three different feature abstraction levels. DeepVA effectively hastened interactive convergence between cognitive understanding and computational modeling of the data, especially in high abstraction tasks.