# Constructive Interpretability with CoLabel: Corroborative Integration, Complementary Features, and Collaborative Learning

arXiv: https://arxiv.org/abs/2205.10011 · HTML: https://mlsanity.com/paper/2205.10011/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 26 of 26)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Person Re-identification: Past, Present and Future](https://mlsanity.com/paper/1610.02984/) (2016)
- [A Large-Scale Car Dataset for Fine-Grained Categorization and Verification](https://mlsanity.com/paper/1506.08959/) (2015)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [Features for Multi-Target Multi-Camera Tracking and Re-Identification](https://mlsanity.com/paper/1803.10859/) (2018)
- [Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression](https://mlsanity.com/paper/1802.06816/) (2018)
- [Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations](https://mlsanity.com/paper/1904.02323/) (2019)
- [An Interpretable Model with Globally Consistent Explanations for Credit Risk](https://mlsanity.com/paper/1811.12615/) (2018)
- [RAM: A Region-Aware Deep Model for Vehicle Re-Identification](https://mlsanity.com/paper/1806.09283/) (2018)
- [Explainable Person Re-Identification with Attribute-guided Metric Distillation](https://mlsanity.com/paper/2103.01451/) (2021)
- [BoxCars: Improving Fine-Grained Recognition of Vehicles using 3-D Bounding Boxes in Traffic Surveillance](https://mlsanity.com/paper/1703.00686/) (2017)
- [Adapting Grad-CAM for Embedding Networks](https://mlsanity.com/paper/2001.06538/) (2020)
- [Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1902.02771/) (2019)
- [Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud](https://mlsanity.com/paper/2109.03285/) (2021)
- [Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision](https://mlsanity.com/paper/2203.13270/) (2022)
- [Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks](https://mlsanity.com/paper/2009.02608/) (2020)
- [LOGO-Net: Large-scale Deep Logo Detection and Brand Recognition with Deep Region-based Convolutional Networks](https://mlsanity.com/paper/1511.02462/) (2015)
- [Looking GLAMORous: Vehicle Re-Id in Heterogeneous Cameras Networks with Global and Local Attention](https://mlsanity.com/paper/2002.02256/) (2020)
- [ODIN: Automated Drift Detection and Recovery in Video Analytics](https://mlsanity.com/paper/2009.05440/) (2020)
- [MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection](https://mlsanity.com/paper/2205.09817/) (2022)

## Abstract (arXiv)

Machine learning models with explainable predictions are increasingly sought after, especially for real-world, mission-critical applications that require bias detection and risk mitigation. Inherent interpretability, where a model is designed from the ground-up for interpretability, provides intuitive insights and transparent explanations on model prediction and performance. In this paper, we present CoLabel, an approach to build interpretable models with explanations rooted in the ground truth. We demonstrate CoLabel in a vehicle feature extraction application in the context of vehicle make-model recognition (VMMR). CoLabel performs VMMR with a composite of interpretable features such as vehicle color, type, and make, all based on interpretable annotations of the ground truth labels. First, CoLabel performs corroborative integration to join multiple datasets that each have a subset of desired annotations of color, type, and make. Then, CoLabel uses decomposable branches to extract complementary features corresponding to desired annotations. Finally, CoLabel fuses them together for final predictions. During feature fusion, CoLabel harmonizes complementary branches so that VMMR features are compatible with each other and can be projected to the same semantic space for classification. With inherent interpretability, CoLabel achieves superior performance to the state-of-the-art black-box models, with accuracy of 0.98, 0.95, and 0.94 on CompCars, Cars196, and BoxCars116K, respectively. CoLabel provides intuitive explanations due to constructive interpretability, and subsequently achieves high accuracy and usability in mission-critical situations.