mlsanityData to 7 Oct 2026

Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models

arXiv 2403.07733 ↗First posted 12 Mar 2024cs.CV cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

LIME (Local Interpretable Model-agnostic Explanations) is a popular XAI framework for unraveling decision-making processes in vision machine-learning models. The technique utilizes image segmentation methods to identify fixed regions for calculating feature importance scores as explanations. Therefore, poor segmentation can weaken the explanation and reduce the importance of segments, ultimately affecting the overall clarity of interpretation. To address these challenges, we introduce the DSEG-LIME (Data-Driven Segmentation LIME) framework, featuring: i) a data-driven segmentation for human-recognized feature generation by foundation model integration, and ii) a user-steered granularity in the hierarchical segmentation procedure through composition. Our findings demonstrate that DSEG outperforms on several XAI metrics on pre-trained ImageNet models and improves the alignment of explanations with human-recognized concepts. The code is available under: https://github. com/patrick-knab/DSEG-LIME

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 39 of its 64 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

39 of 64 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023Segment Anythingcited by 8,158
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022A ConvNet for the 2020scited by 3,545
  12. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
27 more
  1. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  2. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  3. 2020Parameterized Explainer for Graph Neural Networkcited by 423
  4. 2019Counterfactual Visual Explanationscited by 312
  5. 2018On the Robustness of Interpretability Methodscited by 310
  6. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  7. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  8. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  9. 2020Debugging Tests for Model Explanationscited by 146
  10. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  11. 2019XRAI: Better Attributions Through Regionscited by 116
  12. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  13. 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
  14. 2022Deep Hierarchical Semantic Segmentationcited by 84
  15. 2020General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Modelscited by 74
  16. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  17. 2021S-LIME: Stabilized-LIME for Model Explanationcited by 52
  18. 2020BayLIME: Bayesian Local Interpretable Model-Agnostic Explanationscited by 51
  19. 2022Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformerscited by 39
  20. 2023Explain Any Concept: Segment Anything Meets Concept-Based Explanationcited by 37
  21. 2023SLiMe: Segment Like Mecited by 34
  22. 2019Grid Saliency for Context Explanations of Semantic Segmentationcited by 32
  23. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  24. 2020Model Agnostic Multilevel Explanationscited by 24
  25. 2021What does LIME really see in images?cited by 21
  26. 2023GLIME: General, Stable and Local LIME Explanationcited by 21
  27. 2019Effect of Superpixel Aggregation on Explanations in LIME -- A Case Study with Biological Datacited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 3.

8 papers.

  1. 2024DCBM: Data-Efficient Visual Concept Bottleneck Modelscited by 16
  2. 2024Causal Explanations for Image Classifierscited by 9
  3. 2025Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILEcited by 4
  4. 2026ShapBPT: Image Feature Attributions Using Data-Aware Binary Partition Treescited by 2
  5. 2024Interpreting Outliers in Time Series Data through Decoding Autoencodercited by 1
  6. 2025Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classificationcited by 1
  7. 2026Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentationcited by 0
  8. 2026When Superpixels Fail on Documents: A Study of Segmentation for LIME Explanationscited by 0