# Learning Interpretable Concept-Based Models with Human Feedback

arXiv: https://arxiv.org/abs/2012.02898 · HTML: https://mlsanity.com/paper/2012.02898/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2020.

## Built on by (most cited first, 17 of 17)

- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Concept-level Debugging of Part-Prototype Networks](https://mlsanity.com/paper/2205.15769/) (2022)
- [Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents](https://mlsanity.com/paper/2401.05821/) (2024)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)
- [Neural Concept Binder](https://mlsanity.com/paper/2406.09949/) (2024)
- [Discovering User-Interpretable Capabilities of Black-Box Planning Agents](https://mlsanity.com/paper/2107.13668/) (2021)
- [Human-AI Co-design for Clinical Prediction Models](https://mlsanity.com/paper/2601.09072/) (2026)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems](https://mlsanity.com/paper/2402.06287/) (2024)
- [Autonomous Capability Assessment of Sequential Decision-Making Systems in Stochastic Settings (Extended Version)](https://mlsanity.com/paper/2306.04806/) (2023)
- [LICORICE: Label-Efficient Concept-Based Interpretable Reinforcement Learning](https://mlsanity.com/paper/2407.15786/) (2024)
- [Toward a Unified Framework for Debugging Concept-based Models](https://mlsanity.com/paper/2109.11160/) (2021)
- [Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning](https://mlsanity.com/paper/2402.14726/) (2024)
- [Advancing Ante-Hoc Explainable Models through Generative Adversarial Networks](https://mlsanity.com/paper/2401.04647/) (2024)
- [I-CEE: Tailoring Explanations of Image Classification Models to User Expertise](https://mlsanity.com/paper/2312.12102/) (2023)
- [FI-CBL: A Probabilistic Method for Concept-Based Learning with Expert Rules](https://mlsanity.com/paper/2406.19897/) (2024)

## Abstract (arXiv)

Machine learning models that first learn a representation of a domain in terms of human-understandable concepts, then use it to make predictions, have been proposed to facilitate interpretation and interaction with models trained on high-dimensional data. However these methods have important limitations: the way they define concepts are not inherently interpretable, and they assume that concept labels either exist for individual instances or can easily be acquired from users. These limitations are particularly acute for high-dimensional tabular features. We propose an approach for learning a set of transparent concept definitions in high-dimensional tabular data that relies on users labeling concept features instead of individual instances. Our method produces concepts that both align with users' intuitive sense of what a concept means, and facilitate prediction of the downstream label by a transparent machine learning model. This ensures that the full model is transparent and intuitive, and as predictive as possible given this constraint. We demonstrate with simulated user feedback on real prediction problems, including one in a clinical domain, that this kind of direct feedback is much more efficient at learning solutions that align with ground truth concept definitions than alternative transparent approaches that rely on labeling instances or other existing interaction mechanisms, while maintaining similar predictive performance.