# Zero-shot Concept Bottleneck Models

arXiv: https://arxiv.org/abs/2502.09018 · HTML: https://mlsanity.com/paper/2502.09018/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Data Filtering Networks](https://mlsanity.com/paper/2309.17425/) (2023)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)
- [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024)
- [Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery](https://mlsanity.com/paper/2407.14499/) (2024)
- [VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance](https://mlsanity.com/paper/2408.01432/) (2024)
- [A Closer Look at the Intervention Procedure of Concept Bottleneck Models](https://mlsanity.com/paper/2302.14260/) (2023)
- [Text-To-Concept (and Back) via Cross-Model Alignment](https://mlsanity.com/paper/2305.06386/) (2023)
- [Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off](https://mlsanity.com/paper/2209.09056/) (2022)
- [Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability](https://mlsanity.com/paper/2207.09615/) (2022)
- [Vocabulary-free Image Classification](https://mlsanity.com/paper/2306.00917/) (2023)
- [Sparse Linear Concept Discovery Models](https://mlsanity.com/paper/2308.10782/) (2023)
- [Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Concepts](https://mlsanity.com/paper/2408.02265/) (2024)
- [Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization](https://mlsanity.com/paper/1311.5750/) (2013)
- [skscope: Fast Sparsity-Constrained Optimization in Python](https://mlsanity.com/paper/2403.18540/) (2024)

## Built on by (most cited first, 4 of 4)

- [Object Centric Concept Bottlenecks](https://mlsanity.com/paper/2505.24492/) (2025)
- [Explaining CLIP Zero-shot Predictions Through Concepts](https://mlsanity.com/paper/2603.28211/) (2026)
- [Sparse Concept Anchoring for Interpretable and Controllable Neural Representations](https://mlsanity.com/paper/2512.12469/) (2025)
- [Aperture: Training-Free Multiscale Concept Bottlenecks for Remote Sensing](https://mlsanity.com/paper/2609.38603/) (2026)

## Abstract (arXiv)

Concept bottleneck models (CBMs) are inherently interpretable and intervenable neural network models, which explain their final label prediction by the intermediate prediction of high-level semantic concepts. However, they require target task training to learn input-to-concept and concept-to-label mappings, incurring target dataset collections and training resources. In this paper, we present zero-shot concept bottleneck models (Z-CBMs), which predict concepts and labels in a fully zero-shot manner without training neural networks. Z-CBMs utilize a large-scale concept bank, which is composed of millions of vocabulary extracted from the web, to describe arbitrary input in various domains. For the input-to-concept mapping, we introduce concept retrieval, which dynamically finds input-related concepts by the cross-modal search on the concept bank. In the concept-to-label inference, we apply concept regression to select essential concepts from the retrieved concepts by sparse linear regression. Through extensive experiments, we confirm that our Z-CBMs provide interpretable and intervenable concepts without any additional training. Code will be available at https://github.com/yshinya6/zcbm.