# Object Centric Concept Bottlenecks

arXiv: https://arxiv.org/abs/2505.24492 · HTML: https://mlsanity.com/paper/2505.24492/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 60 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 30 of 49)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition](https://mlsanity.com/paper/2001.02407/) (2020)
- [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020)
- [Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery](https://mlsanity.com/paper/2407.14499/) (2024)
- [Concept Bottleneck Model with Additional Unsupervised Concepts](https://mlsanity.com/paper/2202.01459/) (2022)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [Incremental Residual Concept Bottleneck Models](https://mlsanity.com/paper/2404.08978/) (2024)
- [Concept Bottleneck Large Language Models](https://mlsanity.com/paper/2412.07992/) (2024)
- [Interpreting Pretrained Language Models via Concept Bottlenecks](https://mlsanity.com/paper/2311.05014/) (2023)
- [Coarse-to-Fine Concept Bottleneck Models](https://mlsanity.com/paper/2310.02116/) (2023)
- [Interpretable by Design: Learning Predictors by Composing Interpretable Queries](https://mlsanity.com/paper/2207.00938/) (2022)

## Built on by (most cited first, 10 of 10)

- [Synthesizing Visual Concepts as Vision-Language Programs](https://mlsanity.com/paper/2511.18964/) (2025)
- [Neurosymbolic Object-Centric Learning with Distant Supervision](https://mlsanity.com/paper/2506.16129/) (2025)
- [Prototype-Grounded Concept Models for Verifiable Concept Alignment](https://mlsanity.com/paper/2604.16076/) (2026)
- [Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations](https://mlsanity.com/paper/2605.16405/) (2026)
- [Caption Bottleneck Models](https://mlsanity.com/paper/2607.00578/) (2026)
- [Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction](https://mlsanity.com/paper/2607.08233/) (2026)
- [Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings](https://mlsanity.com/paper/2507.07532/) (2025)
- [GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning](https://mlsanity.com/paper/2604.16871/) (2026)
- [Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models](https://mlsanity.com/paper/2606.04326/) (2026)
- [COCOLogic-V2: Identifying Logical Inconsistencies via Truly Hard-Negatives](https://mlsanity.com/paper/2606.28194/) (2026)

## Abstract (arXiv)

Developing high-performing, yet interpretable models remains a critical challenge in modern AI. Concept-based models (CBMs) attempt to address this by extracting human-understandable concepts from a global encoding (e.g., image encoding) and then applying a linear classifier on the resulting concept activations, enabling transparent decision-making. However, their reliance on holistic image encodings limits their expressiveness in object-centric real-world settings and thus hinders their ability to solve complex vision tasks beyond single-label classification. To tackle these challenges, we introduce Object-Centric Concept Bottlenecks (OCB), a framework that combines the strengths of CBMs and pre-trained object-centric foundation models, boosting performance and interpretability. We evaluate OCB on complex image datasets and conduct a comprehensive ablation study to analyze key components of the framework, such as strategies for aggregating object-concept encodings. The results show that OCB outperforms traditional CBMs and allows one to make interpretable decisions for complex visual tasks.