# Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos

arXiv: https://arxiv.org/abs/2309.08022 · HTML: https://mlsanity.com/paper/2309.08022/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022)
- [Just Ask: Learning to Answer Questions from Millions of Narrated Videos](https://mlsanity.com/paper/2012.00451/) (2020)
- [mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections](https://mlsanity.com/paper/2205.12005/) (2022)
- [Revealing Single Frame Bias for Video-and-Language Learning](https://mlsanity.com/paper/2206.03428/) (2022)
- [Flow-Guided Sparse Transformer for Video Deblurring](https://mlsanity.com/paper/2201.01893/) (2022)
- [Why Does a Visual Question Have Different Answers?](https://mlsanity.com/paper/1908.04342/) (2019)
- [ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition](https://mlsanity.com/paper/2104.03841/) (2021)
- [Assessing Image Quality Issues for Real-World Problems](https://mlsanity.com/paper/2003.12511/) (2020)
- [Semantic Segmentation using Neural Ordinary Differential Equations](https://mlsanity.com/paper/2209.08667/) (2022)

## Built on by (most cited first, 6 of 6)

- [Sentence Attention Blocks for Answer Grounding](https://mlsanity.com/paper/2309.11593/) (2023)
- [Improving Normalization with the James-Stein Estimator](https://mlsanity.com/paper/2312.00313/) (2023)
- [On the Effect of Image Resolution on Semantic Segmentation](https://mlsanity.com/paper/2402.05398/) (2024)
- [FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimator](https://mlsanity.com/paper/2410.03499/) (2024)
- [Multi-Scale Semantic Segmentation with Modified MBConv Blocks](https://mlsanity.com/paper/2402.04618/) (2024)
- [Early Fusion of Features for Semantic Segmentation](https://mlsanity.com/paper/2402.06091/) (2024)

## Abstract (arXiv)

Numerous applications have been developed to assist visually impaired individuals that employ a machine learning unit to process visual input. However, a critical challenge with these applications is the sub-optimal quality of images captured by the users. Given the complexity of operating a camera for visually impaired individuals, we advocate for the use of Apple Live Photos and Android Motion Photos technologies. In this study, we introduce a straightforward methodology to evaluate and contrast the efficacy of Live/Motion Photos against traditional image-based approaches. Our findings reveal that both Live Photos and Motion Photos outperform single-frame images in common visual assisting tasks, specifically in object classification and VideoQA. We validate our results through extensive experiments on the ORBIT dataset, which consists of videos collected by visually impaired individuals. Furthermore, we conduct a series of ablation studies to delve deeper into the impact of deblurring and longer temporal crops.