# Semantic bottleneck for computer vision tasks

arXiv: https://arxiv.org/abs/1811.02234 · HTML: https://mlsanity.com/paper/1811.02234/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2018.

## Built on by (most cited first, 10 of 10)

- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Learning Bottleneck Concepts in Image Classification](https://mlsanity.com/paper/2304.10131/) (2023)
- [Static and Dynamic Concepts for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2207.12795/) (2022)
- [Interpretable Prognostics with Concept Bottleneck Models](https://mlsanity.com/paper/2405.17575/) (2024)
- [CAT: Interpretable Concept-based Taylor Additive Models](https://mlsanity.com/paper/2406.17931/) (2024)
- [Interpretable Measures of Conceptual Similarity by Complexity-Constrained Descriptive Auto-Encoding](https://mlsanity.com/paper/2402.08919/) (2024)
- [Image Classification with Consistent Supporting Evidence](https://mlsanity.com/paper/2111.07048/) (2021)
- [Automated Molecular Concept Generation and Labeling with Large Language Models](https://mlsanity.com/paper/2406.09612/) (2024)
- [The Definitions of Interpretability and Learning of Interpretable Models](https://mlsanity.com/paper/2105.14171/) (2021)
- [Enhancing Performance of Explainable AI Models with Constrained Concept Refinement](https://mlsanity.com/paper/2502.06775/) (2025)

## Abstract (arXiv)

This paper introduces a novel method for the representation of images that is semantic by nature, addressing the question of computation intelligibility in computer vision tasks. More specifically, our proposition is to introduce what we call a semantic bottleneck in the processing pipeline, which is a crossing point in which the representation of the image is entirely expressed with natural language , while retaining the efficiency of numerical representations. We show that our approach is able to generate semantic representations that give state-of-the-art results on semantic content-based image retrieval and also perform very well on image classification tasks. Intelligibility is evaluated through user centered experiments for failure detection.