# "Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications

arXiv: https://arxiv.org/abs/2503.03927 · HTML: https://mlsanity.com/paper/2503.03927/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 116 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 23 of 23)

- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- ["Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction](https://mlsanity.com/paper/2210.03735/) (2022)
- [Design Principles for Generative AI Applications](https://mlsanity.com/paper/2401.14484/) (2024)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps](https://mlsanity.com/paper/2002.07687/) (2020)
- [Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI](https://mlsanity.com/paper/2302.00799/) (2023)
- [Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks](https://mlsanity.com/paper/2310.07879/) (2023)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study](https://mlsanity.com/paper/2309.10108/) (2023)
- [Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Apps](https://mlsanity.com/paper/2101.04401/) (2021)
- [Transparent AI Disclosure Obligations: Who, What, When, Where, Why, How](https://mlsanity.com/paper/2403.06823/) (2024)
- [Deep Learning Towards Mobile Applications](https://mlsanity.com/paper/1809.03559/) (2018)
- [Understanding Real-world Threats to Deep Learning Models in Android Apps](https://mlsanity.com/paper/2209.09577/) (2022)
- [Smart App Attack: Hacking Deep Learning Models in Android Apps](https://mlsanity.com/paper/2204.11075/) (2022)
- [Crowdsourcing the Perception of Machine Teaching](https://mlsanity.com/paper/2002.01618/) (2020)
- [Machine Learning Processes as Sources of Ambiguity: Insights from AI Art](https://mlsanity.com/paper/2403.09374/) (2024)
- [Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit](https://mlsanity.com/paper/2402.13437/) (2024)
- [Mapping the Design Space of Teachable Social Media Feed Experiences](https://mlsanity.com/paper/2401.14000/) (2024)
- [Vis Ex Machina: An Analysis of Trust in Human versus Algorithmically Generated Visualization Recommendations](https://mlsanity.com/paper/2101.04251/) (2021)
- [Understanding Older Adults' Perceptions and Challenges in Using AI-enabled Everyday Technologies](https://mlsanity.com/paper/2210.01369/) (2022)
- [A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTok](https://mlsanity.com/paper/2403.19717/) (2024)
- [Yourfeed: Towards open science and interoperable systems for social media](https://mlsanity.com/paper/2207.07478/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Meaningful Transparency in Civic AI Systems](https://mlsanity.com/paper/2510.07889/) (2025)

## Abstract (arXiv)

Machine learning models deployed locally on social media applications are used for features, such as face filters which read faces in-real time, and they expose sensitive attributes to the apps. However, the deployment of machine learning models, e.g., when, where, and how they are used, in social media applications is opaque to users. We aim to address this inconsistency and investigate how social media user perceptions and behaviors change once exposed to these models. We conducted user studies (N=21) and found that participants were unaware to both what the models output and when the models were used in Instagram and TikTok, two major social media platforms. In response to being exposed to the models' functionality, we observed long term behavior changes in 8 participants. Our analysis uncovers the challenges and opportunities in providing transparency for machine learning models that interact with local user data.