# Feedback is Needed for Retakes: An Explainable Poor Image Notification Framework for the Visually Impaired

arXiv: https://arxiv.org/abs/2211.09427 · HTML: https://mlsanity.com/paper/2211.09427/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [A Comprehensive Survey of Deep Learning for Image Captioning](https://mlsanity.com/paper/1810.04020/) (2018)
- [Captioning Images Taken by People Who Are Blind](https://mlsanity.com/paper/2002.08565/) (2020)
- [Assessing Image Quality Issues for Real-World Problems](https://mlsanity.com/paper/2003.12511/) (2020)
- [Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge](https://mlsanity.com/paper/2012.11696/) (2020)
- [Smart Guiding Glasses for Visually Impaired People in Indoor Environment](https://mlsanity.com/paper/1709.09359/) (2017)
- [Multi-Modal Image Captioning for the Visually Impaired](https://mlsanity.com/paper/2105.08106/) (2021)
- [Inverse Problems Leveraging Pre-trained Contrastive Representations](https://mlsanity.com/paper/2110.07439/) (2021)
- [DeepSeek: Content Based Image Search & Retrieval](https://mlsanity.com/paper/1801.03406/) (2018)

## Abstract (arXiv)

We propose a simple yet effective image captioning framework that can determine the quality of an image and notify the user of the reasons for any flaws in the image. Our framework first determines the quality of images and then generates captions using only those images that are determined to be of high quality. The user is notified by the flaws feature to retake if image quality is low, and this cycle is repeated until the input image is deemed to be of high quality. As a component of the framework, we trained and evaluated a low-quality image detection model that simultaneously learns difficulty in recognizing images and individual flaws, and we demonstrated that our proposal can explain the reasons for flaws with a sufficient score. We also evaluated a dataset with low-quality images removed by our framework and found improved values for all four common metrics (e.g., BLEU-4, METEOR, ROUGE-L, CIDEr), confirming an improvement in general-purpose image captioning capability. Our framework would assist the visually impaired, who have difficulty judging image quality.