# Demonstration of MaskSearch: Efficiently Querying Image Masks for Machine Learning Workflows

arXiv: https://arxiv.org/abs/2404.06563 · HTML: https://mlsanity.com/paper/2404.06563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 9 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?](https://mlsanity.com/paper/1606.03556/) (2016)
- [The iWildCam 2020 Competition Dataset](https://mlsanity.com/paper/2004.10340/) (2020)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Finding and Fixing Spurious Patterns with Explanations](https://mlsanity.com/paper/2106.02112/) (2021)
- [Human Attention in Fine-grained Classification](https://mlsanity.com/paper/2111.01628/) (2021)
- [Detection Defense Against Adversarial Attacks with Saliency Map](https://mlsanity.com/paper/2009.02738/) (2020)
- [DeepBase: Deep Inspection of Neural Networks](https://mlsanity.com/paper/1808.04486/) (2018)
- [DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation](https://mlsanity.com/paper/2104.02234/) (2021)

## Abstract (arXiv)

We demonstrate MaskSearch, a system designed to accelerate queries over databases of image masks generated by machine learning models. MaskSearch formalizes and accelerates a new category of queries for retrieving images and their corresponding masks based on mask properties, which support various applications, from identifying spurious correlations learned by models to exploring discrepancies between model saliency and human attention. This demonstration makes the following contributions:(1) the introduction of MaskSearch's graphical user interface (GUI), which enables interactive exploration of image databases through mask properties, (2) hands-on opportunities for users to explore MaskSearch's capabilities and constraints within machine learning workflows, and (3) an opportunity for conference attendees to understand how MaskSearch accelerates queries over image masks.