# Interactive Attention AI to translate low light photos to captions for night scene understanding in women safety

arXiv: https://arxiv.org/abs/2201.00969 · HTML: https://mlsanity.com/paper/2201.00969/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 4 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [SimVLM: Simple Visual Language Model Pretraining with Weak Supervision](https://mlsanity.com/paper/2108.10904/) (2021)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation](https://mlsanity.com/paper/2103.00860/) (2021)
- [Low-Light Image and Video Enhancement Using Deep Learning: A Survey](https://mlsanity.com/paper/2104.10729/) (2021)
- [Captioning Images Taken by People Who Are Blind](https://mlsanity.com/paper/2002.08565/) (2020)
- [Object-and-Action Aware Model for Visual Language Navigation](https://mlsanity.com/paper/2007.14626/) (2020)
- [On-Device Machine Learning: An Algorithms and Learning Theory Perspective](https://mlsanity.com/paper/1911.00623/) (2019)
- [Convolutional Image Captioning](https://mlsanity.com/paper/1711.09151/) (2017)
- [Variational Attention for Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.08207/) (2017)
- [iCap: Interactive Image Captioning with Predictive Text](https://mlsanity.com/paper/2001.11782/) (2020)

## Abstract (arXiv)

There is amazing progress in Deep Learning based models for Image captioning and Low Light image enhancement. For the first time in literature, this paper develops a Deep Learning model that translates night scenes to sentences, opening new possibilities for AI applications in the safety of visually impaired women. Inspired by Image Captioning and Visual Question Answering, a novel Interactive Image Captioning is developed. A user can make the AI focus on any chosen person of interest by influencing the attention scoring. Attention context vectors are computed from CNN feature vectors and user-provided start word. The Encoder-Attention-Decoder neural network learns to produce captions from low brightness images. This paper demonstrates how women safety can be enabled by researching a novel AI capability in the Interactive Vision-Language model for perception of the environment in the night.