mlsanityData to 7 Oct 2026

Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos

arXiv 2309.08022 ↗First posted 14 Sep 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Numerous applications have been developed to assist visually impaired individuals that employ a machine learning unit to process visual input. However, a critical challenge with these applications is the sub-optimal quality of images captured by the users. Given the complexity of operating a camera for visually impaired individuals, we advocate for the use of Apple Live Photos and Android Motion Photos technologies. In this study, we introduce a straightforward methodology to evaluate and contrast the efficacy of Live/Motion Photos against traditional image-based approaches. Our findings reveal that both Live Photos and Motion Photos outperform single-frame images in common visual assisting tasks, specifically in object classification and VideoQA. We validate our results through extensive experiments on the ORBIT dataset, which consists of videos collected by visually impaired individuals. Furthermore, we conduct a series of ablation studies to delve deeper into the impact of deblurring and longer temporal crops.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 14 of its 21 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

14 of 21 references.

References · 14

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021ViViT: A Video Vision Transformercited by 1,610
  3. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  4. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  5. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  6. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  7. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  8. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  9. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 131
  10. 2022Flow-Guided Sparse Transformer for Video Deblurringcited by 43
  11. 2019Why Does a Visual Question Have Different Answers?cited by 34
  12. 2021ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognitioncited by 29
2 more
  1. 2020Assessing Image Quality Issues for Real-World Problemscited by 28
  2. 2022Semantic Segmentation using Neural Ordinary Differential Equationscited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4.

6 papers.

  1. 2023Sentence Attention Blocks for Answer Groundingcited by 8
  2. 2023Improving Normalization with the James-Stein Estimatorcited by 6
  3. 2024On the Effect of Image Resolution on Semantic Segmentationcited by 1
  4. 2024FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimatorcited by 1
  5. 2024Multi-Scale Semantic Segmentation with Modified MBConv Blockscited by 0
  6. 2024Early Fusion of Features for Semantic Segmentationcited by 0