# How to make effective use of domain experts for image classification?

arXiv: https://arxiv.org/abs/2609.17749 · HTML: https://mlsanity.com/paper/2609.17749/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Interactive Concept Bottleneck Models](https://mlsanity.com/paper/2212.07430/) (2022)
- [Concept-level Debugging of Part-Prototype Networks](https://mlsanity.com/paper/2205.15769/) (2022)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Advanced wood species identification based on multiple anatomical sections and using deep feature transfer and fusion](https://mlsanity.com/paper/2404.08585/) (2024)

## Abstract (arXiv)

A lot of expectations have been put for years on integrating domain expert knowledge in image classification models. Several approaches have been explored, Concept Bottleneck Models (CBMs) opened up a new avenue of research leading to many variants, and more recently to Concept-based Embedding Models (CEMs). CBM consider binary encoding of each concept, while CEM expands this idea by embedding each concept through two vectors. However in real-life scenarii, domain experts' knowledge is usually organized in concepts determined by various attributes, each attribute encoded either with numerical values, or range of values, or binary values, or categorical values. In this work, we first finetune an image feature extractor for classifying attributes representing the downstream object classes, where the class attributes have been specified by experts under various encoding formats. A classification head is then learnt from these various attributes to categorize target objects. We experimentally show that it improves the classification for three datasets: Kaggle fish dataset, AWA2 and a more challenging new wood charcoal dataset. We then propose an automatic selection of potential missclassified data. In this second step, experts are asked for those data to eventually modify the predicted attributes in order to improve the classification.