mlsanityData to 7 Oct 2026

SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions

arXiv 2507.23784 ↗First posted 31 Jul 2025cs.CV cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Concept Bottleneck Models (CBMs) and other concept-based interpretable models show great promise for making AI applications more transparent, which is essential in fields like medicine. Despite their success, we demonstrate that CBMs struggle to reliably identify the correct concepts under distribution shifts. To assess the robustness of CBMs to concept variations, we introduce SUB: a fine-grained image and concept benchmark containing 38,400 synthetic images based on the CUB dataset. To create SUB, we select a CUB subset of 33 bird classes and 45 concepts to generate images which substitute a specific concept, such as wing color or belly pattern. We introduce a novel Tied Diffusion Guidance (TDG) method to precisely control generated images, where noise sharing for two parallel denoising processes ensures that both the correct bird class and the correct attribute are generated. This novel benchmark enables rigorous evaluation of CBMs and similar interpretable models, contributing to the development of more robust methods. Our code is available at https://github.com/ExplainableML/sub and the dataset at http://huggingface.co/datasets/Jessica-bader/SUB.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 60 of its 71 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

60 of 71 references.

References · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  11. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  12. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
38 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  2. 2020Concept Bottleneck Modelscited by 1,022
  3. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  4. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  8. 2022Visual Classification via Description from Large Language Modelscited by 331
  9. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  10. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  11. 2023Label-Free Concept Bottleneck Modelscited by 282
  12. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  13. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  14. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 128
  15. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  16. 2021Do Concept Bottleneck Models Learn as Intended?cited by 114
  17. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  18. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  19. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  20. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  21. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 89
  22. 2022Concept Bottleneck Model with Additional Unsupervised Conceptscited by 79
  23. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
  24. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  25. 2021Learning to Compose Visual Relationscited by 72
  26. 2023Compositional Generalization from First Principlescited by 62
  27. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  28. 2022Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Offcited by 61
  29. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 49
  30. 2023Coarse-to-Fine Concept Bottleneck Modelscited by 31
  31. 2023Towards Robust Metrics for Concept Representation Evaluationcited by 29
  32. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion modelscited by 29
  33. 2024On Discrete Prompt Optimization for Diffusion Modelscited by 28
  34. 2024DataDream: Few-shot Guided Dataset Generationcited by 28
  35. 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Modelscited by 23
  36. 2022Understanding and Enhancing Robustness of Concept-based Modelscited by 22
  37. 2023PDiscoNet: Semantically consistent part discovery for fine-grained recognitioncited by 21
  38. 2024Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Conceptscited by 20

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Stitch: Training-Free Position Control in Multimodal Diffusion Transformerscited by 2
  2. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotationscited by 2
  3. 2026Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformerscited by 1
  4. 2026Logit Distance Bounds Representational Similaritycited by 0
  5. 2026The Latent Color Subspace: Emergent Order in High-Dimensional Chaoscited by 0
  6. 2026Metonymy in vision models undermines attention-based interpretabilitycited by 0