# LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

arXiv: https://arxiv.org/abs/2402.12525 · HTML: https://mlsanity.com/paper/2402.12525/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [Eigen-CAM: Class Activation Map using Principal Components](https://mlsanity.com/paper/2008.00299/) (2020)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks](https://mlsanity.com/paper/2311.01361/) (2023)
- [PromptCap: Prompt-Guided Task-Aware Image Captioning](https://mlsanity.com/paper/2211.09699/) (2022)
- [Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks](https://mlsanity.com/paper/2011.08891/) (2020)
- [Black-box Explanation of Object Detectors via Saliency Maps](https://mlsanity.com/paper/2006.03204/) (2020)
- [TTPLA: An Aerial-Image Dataset for Detection and Segmentation of Transmission Towers and Power Lines](https://mlsanity.com/paper/2010.10032/) (2020)
- [Towards Better Explanations for Object Detection](https://mlsanity.com/paper/2306.02744/) (2023)
- [Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networks](https://mlsanity.com/paper/2102.07799/) (2021)
- [EUCA: the End-User-Centered Explainable AI Framework](https://mlsanity.com/paper/2102.02437/) (2021)
- [Towards Trust of Explainable AI in Thyroid Nodule Diagnosis](https://mlsanity.com/paper/2303.04731/) (2023)
- [G-CAME: Gaussian-Class Activation Mapping Explainer for Object Detectors](https://mlsanity.com/paper/2306.03400/) (2023)
- [A Novel Explainable Artificial Intelligence Model in Image Classification problem](https://mlsanity.com/paper/2307.04137/) (2023)
- [XAI-Enhanced Semantic Segmentation Models for Visual Quality Inspection](https://mlsanity.com/paper/2401.09900/) (2024)
- [Transforming gradient-based techniques into interpretable methods](https://mlsanity.com/paper/2401.14434/) (2024)

## Built on by (most cited first, 13 of 13)

- [XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approach](https://mlsanity.com/paper/2407.11771/) (2024)
- [Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretation](https://mlsanity.com/paper/2512.08934/) (2025)
- [Heart2Mind: Human-Centered Contestable Psychiatric Disorder Diagnosis System using Wearable ECG Monitors](https://mlsanity.com/paper/2505.11612/) (2025)
- [Multimedia Verification Through Multi-Agent Deep Research Multimodal Large Language Models](https://mlsanity.com/paper/2507.04410/) (2025)
- [ODExAI: A Comprehensive Object Detection Explainable AI Evaluation](https://mlsanity.com/paper/2504.19249/) (2025)
- [SimInterview: Transforming Business Education through Large Language Model-Based Simulated Multilingual Interview Training System](https://mlsanity.com/paper/2508.11873/) (2025)
- [Advancing Histopathology-Based Breast Cancer Diagnosis: Insights into Multi-Modality and Explainability](https://mlsanity.com/paper/2406.12897/) (2024)
- [A Multi-Task Text Classification Pipeline with Natural Language Explanations: A User-Centric Evaluation in Sentiment Analysis and Offensive Language Identification in Greek Tweets](https://mlsanity.com/paper/2410.10290/) (2024)
- [GeoVision Labeler: Zero-Shot Geospatial Classification with Vision and Language Models](https://mlsanity.com/paper/2505.24340/) (2025)
- [A Novel Framework for Automated Explain Vision Model Using Vision-Language Models](https://mlsanity.com/paper/2508.20227/) (2025)
- [Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation](https://mlsanity.com/paper/2602.15650/) (2026)
- [Multi-Dimensional Model Integrity and Responsibility Assessment Index and Scoring Framework](https://mlsanity.com/paper/2605.14550/) (2026)
- [Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detection](https://mlsanity.com/paper/2609.31690/) (2026)

## Abstract (arXiv)

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.