# VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis

arXiv: https://arxiv.org/abs/2505.03132 · HTML: https://mlsanity.com/paper/2505.03132/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 61 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Simple data balancing achieves competitive worst-group-accuracy](https://mlsanity.com/paper/2110.14503/) (2021)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Explaining NLP Models via Minimal Contrastive Editing (MiCE)](https://mlsanity.com/paper/2012.13985/) (2020)
- [ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions](https://mlsanity.com/paper/2303.06594/) (2023)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey](https://mlsanity.com/paper/2112.08936/) (2021)
- [Distilling Model Failures as Directions in Latent Space](https://mlsanity.com/paper/2206.14754/) (2022)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Object detection via a multi-region & semantic segmentation-aware CNN model](https://mlsanity.com/paper/1505.01749/) (2015)
- [Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs](https://mlsanity.com/paper/2010.07526/) (2020)
- [Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning](https://mlsanity.com/paper/2208.08831/) (2022)
- [ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations](https://mlsanity.com/paper/2211.01866/) (2022)
- [Adaptive Testing of Computer Vision Models](https://mlsanity.com/paper/2212.02774/) (2022)
- [Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning](https://mlsanity.com/paper/2302.04732/) (2023)
- [PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning](https://mlsanity.com/paper/2308.03977/) (2023)

## Built on by (most cited first, 5 of 5)

- [ReME: A Data-Centric Framework for Training-Free Open-Vocabulary Segmentation](https://mlsanity.com/paper/2506.21233/) (2025)
- [Teaching People LLM's Errors and Getting it Right](https://mlsanity.com/paper/2512.21422/) (2025)
- [AwesomeLit: Towards Hypothesis Generation with Agent-Supported Literature Research](https://mlsanity.com/paper/2603.22648/) (2026)
- [GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks](https://mlsanity.com/paper/2512.24592/) (2025)
- [TopoAlign: Topology-Aware Visual Representation Alignment](https://mlsanity.com/paper/2605.25541/) (2026)

## Abstract (arXiv)

Real-world machine learning models require rigorous evaluation before deployment, especially in safety-critical domains like autonomous driving and surveillance. The evaluation of machine learning models often focuses on data slices, which are subsets of the data that share a set of characteristics. Data slice finding automatically identifies conditions or data subgroups where models underperform, aiding developers in mitigating performance issues. Despite its popularity and effectiveness, data slicing for vision model validation faces several challenges. First, data slicing often needs additional image metadata or visual concepts, and falls short in certain computer vision tasks, such as object detection. Second, understanding data slices is a labor-intensive and mentally demanding process that heavily relies on the expert's domain knowledge. Third, data slicing lacks a human-in-the-loop solution that allows experts to form hypothesis and test them interactively. To overcome these limitations and better support the machine learning operations lifecycle, we introduce VISLIX, a novel visual analytics framework that employs state-of-the-art foundation models to help domain experts analyze slices in computer vision models. Our approach does not require image metadata or visual concepts, automatically generates natural language insights, and allows users to test data slice hypothesis interactively. We evaluate VISLIX with an expert study and three use cases, that demonstrate the effectiveness of our tool in providing comprehensive insights for validating object detection models.