A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTok
Not on the map. Why?
Abstract (arXiv)
Mobile apps have embraced user privacy by moving their data processing to the user's smartphone. Advanced machine learning (ML) models, such as vision models, can now locally analyze user images to extract insights that drive several functionalities. Capitalizing on this new processing model of locally analyzing user images, we analyze two popular social media apps, TikTok and Instagram, to reveal (1) what insights vision models in both apps infer about users from their image and video data and (2) whether these models exhibit performance disparities with respect to demographics. As vision models provide signals for sensitive technologies like age verification and facial recognition, understanding potential biases in these models is crucial for ensuring that users receive equitable and accurate services. We develop a novel method for capturing and evaluating ML tasks in mobile apps, overcoming challenges like code obfuscation, native code execution, and scalability. Our method comprises ML task detection, ML pipeline reconstruction, and ML performance assessment, specifically focusing on demographic disparities. We apply our methodology to TikTok and Instagram, revealing significant insights. For TikTok, we find issues in age and gender prediction accuracy, particularly for minors and Black individuals. In Instagram, our analysis uncovers demographic disparities in the extraction of over 500 visual concepts from images, with evidence of spurious correlations between demographic features and certain concepts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 83 references that are in the map. First posted on arXiv 27 Mar 2024.
Builds on
17 of 83 references.
References · 17
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
- 2020Fairness in Machine Learningcited by 194
- 2021Gender Bias in Machine Translationcited by 116
- 2020CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotationscited by 109
- 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 69
- 2020Fairness Through Robustness: Investigating Robustness Disparity in Deep Learningcited by 55
- 2018A First Look at Deep Learning Apps on Smartphonescited by 47
- 2020Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Appscited by 35
- 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
- 2021On the use of automatically generated synthetic image datasets for benchmarking face recognitioncited by 26
- 2021Smart at what cost? Characterising Mobile Deep Neural Networks in the wildcited by 25
- 2021Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Appscited by 19
5 more
- 2022Understanding Real-world Threats to Deep Learning Models in Android Appscited by 15
- 2022Cross-Language Binary-Source Code Matching with Intermediate Representationscited by 13
- 2022Smart App Attack: Hacking Deep Learning Models in Android Appscited by 12
- 2023Uncovering Bias in Face Generation Modelscited by 9
- 2021JuCify: A Step Towards Android Code Unification for Enhanced Static Analysiscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.