# AnnoSense: A Framework for Physiological Emotion Data Collection in Everyday Settings for AI

arXiv: https://arxiv.org/abs/2508.02680 · HTML: https://mlsanity.com/paper/2508.02680/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 167 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 24 of 24)

- [MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journaling](https://mlsanity.com/paper/2310.05231/) (2023)
- [K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations](https://mlsanity.com/paper/2005.04120/) (2020)
- [The Principles of Data-Centric AI (DCAI)](https://mlsanity.com/paper/2211.14611/) (2022)
- [MindScape Study: Integrating LLM and Behavioral Sensing for Personalized AI-Driven Journaling Experiences](https://mlsanity.com/paper/2409.09570/) (2024)
- [Localized Cultural Knowledge is Conserved and Controllable in Large Language Models](https://mlsanity.com/paper/2504.10191/) (2025)
- [A dataset of continuous affect annotations and physiological signals for emotion analysis](https://mlsanity.com/paper/1812.02782/) (2018)
- [MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Intervention](https://mlsanity.com/paper/2309.16639/) (2023)
- [A multimodal sensor dataset for continuous stress detection of nurses in a hospital](https://mlsanity.com/paper/2108.07689/) (2021)
- [Contextual AI Journaling: Integrating LLM and Time Series Behavioral Sensing Technology to Promote Self-Reflection and Well-being using the MindScape App](https://mlsanity.com/paper/2404.00487/) (2024)
- [Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries](https://mlsanity.com/paper/2211.03009/) (2022)
- [Designing for emotion regulation interventions: an agenda for HCI theory and research](https://mlsanity.com/paper/2204.00118/) (2022)
- [Mirror Ritual: An Affective Interface for Emotional Self-Reflection](https://mlsanity.com/paper/2004.09685/) (2020)
- [Investigating the Reliability of Self-report Data in the Wild: The Quest for Ground Truth](https://mlsanity.com/paper/2107.00389/) (2021)
- [Exploring the Effects of AI-assisted Emotional Support Processes in Online Mental Health Community](https://mlsanity.com/paper/2202.10065/) (2022)
- [Representation Debiasing of Generated Data Involving Domain Experts](https://mlsanity.com/paper/2407.09485/) (2024)
- [SensingKit: Evaluating the Sensor Power Consumption in iOS devices](https://mlsanity.com/paper/1606.05576/) (2016)
- [ForDigitStress: A multi-modal stress dataset employing a digital job interview scenario](https://mlsanity.com/paper/2303.07742/) (2023)
- [DiaryHelper: Exploring the Use of an Automatic Contextual Information Recording Agent for Elicitation Diary Study](https://mlsanity.com/paper/2404.19738/) (2024)
- [Stressor Type Matters! -- Exploring Factors Influencing Cross-Dataset Generalizability of Physiological Stress Detection](https://mlsanity.com/paper/2405.09563/) (2024)
- [DiversityOne: A Multi-Country Smartphone Sensor Dataset for Everyday Life Behavior Modeling](https://mlsanity.com/paper/2502.03347/) (2025)
- [Critiquing Self-report Practices for Human Mental and Wellbeing Computing at Ubicomp](https://mlsanity.com/paper/2311.15496/) (2023)
- [Wearable Healthcare Devices for Monitoring Stress and Attention Level in Workplace Environments](https://mlsanity.com/paper/2406.05813/) (2024)
- [Can we say a cat is a cat? Understanding the challenges in annotating physiological signal-based emotion data](https://mlsanity.com/paper/2406.14908/) (2024)
- [Translating Emotions to Annotations -- A Participant Perspective of Physiological Emotion Data Collection](https://mlsanity.com/paper/2503.19636/) (2025)

## Abstract (arXiv)

Emotional and mental well-being are vital components of quality of life, and with the rise of smart devices like smartphones, wearables, and artificial intelligence (AI), new opportunities for monitoring emotions in everyday settings have emerged. However, for AI algorithms to be effective, they require high-quality data and accurate annotations. As the focus shifts towards collecting emotion data in real-world environments to capture more authentic emotional experiences, the process of gathering emotion annotations has become increasingly complex. This work explores the challenges of everyday emotion data collection from the perspectives of key stakeholders. We collected 75 survey responses, performed 32 interviews with the public, and 3 focus group discussions (FGDs) with 12 mental health professionals. The insights gained from a total of 119 stakeholders informed the development of our framework, AnnoSense, designed to support everyday emotion data collection for AI. This framework was then evaluated by 25 emotion AI experts for its clarity, usefulness, and adaptability. Lastly, we discuss the potential next steps and implications of AnnoSense for future research in emotion AI, highlighting its potential to enhance the collection and analysis of emotion data in real-world contexts.