mlsanityData to 14 Oct 2026

Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps

arXiv 2307.00756 ↗First posted 3 Jul 2023cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

With the emergence of deep learning techniques, smartphone apps are now embedded on-device AI features for enabling advanced tasks like speech translation, to attract users and increase market competitiveness. A good interaction design is important to make an AI feature usable and understandable. However, AI features have their unique challenges like sensitiveness to the input, dynamic behaviours and output uncertainty. Existing guidelines and tools either do not cover AI features or consider mobile apps which are confirmed by our informal interview with professional designers. To address these issues, we conducted the first empirical study to explore user-AI-interaction in mobile apps. We aim to understand the status of on-device AI usage by investigating 176 AI apps from 62,822 apps. We identified 255 AI features and summarised 759 implementations into three primary interaction pattern types. We further implemented our findings into a multi-faceted search-enabled gallery. The results of the user study demonstrate the usefulness of our findings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 44 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

16 of 44 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  6. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  7. 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
  8. 2019On-Device Machine Learning: An Algorithms and Learning Theory Perspectivecited by 63
  9. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  10. 2021Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AIcited by 38
  11. 2020Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Appscited by 36
  12. 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
4 more
  1. 2021Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Appscited by 19
  2. 2022Understanding Real-world Threats to Deep Learning Models in Android Appscited by 15
  3. 2022Smart App Attack: Hacking Deep Learning Models in Android Appscited by 12
  4. 2015Colors -Messengers of Concepts: Visual Design Mining for Learning Color Semanticscited by 7

Built on by

Not cited yet.