# Investigating the Reliability of Self-report Data in the Wild: The Quest for Ground Truth

arXiv: https://arxiv.org/abs/2107.00389 · HTML: https://mlsanity.com/paper/2107.00389/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2021.

## Built on by (most cited first, 5 of 5)

- [Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on Engagement](https://mlsanity.com/paper/2409.13726/) (2024)
- [A Versatile Dataset of Mouse and Eye Movements on Search Engine Results Pages](https://mlsanity.com/paper/2507.08003/) (2025)
- [AnnoSense: A Framework for Physiological Emotion Data Collection in Everyday Settings for AI](https://mlsanity.com/paper/2508.02680/) (2025)
- [Artificial Intelligence can Recognize Whether a Job Applicant is Selling and/or Lying According to Facial Expressions and Head Movements Much More Correctly Than Human Interviewers](https://mlsanity.com/paper/2605.17461/) (2026)
- [LLM-Based Synthetic Ground Truth Generation for Audio-Based Emotion Classification via In-Context Learning](https://mlsanity.com/paper/2606.14784/) (2026)

## Abstract (arXiv)

Inferring human mental state (e.g., emotion, depression, engagement) with sensing technology is one of the most valuable challenges in the affective computing area, which has a profound impact in all industries interacting with humans. The self-report survey is the most common way to quantify how people think, but prone to subjectivity and various responses bias. It is usually used as the ground truth for human mental state prediction. In recent years, many data-driven machine learning models are built based on self-report annotations as the target value. In this research, we investigate the reliability of self-report surveys in the wild by studying the confidence level of responses and survey completion time. We conduct a case study (i.e., student engagement inference) by recruiting 23 students in a high school setting over a period of 4 weeks. Our participants volunteered 488 self-reported responses and data from their wearable sensors. We also find the physiologically measured student engagement and perceived student engagement are not always consistent. The findings from this research have great potential to benefit future studies in predicting engagement, depression, stress, and other emotion-related states in the field of affective computing and sensing technologies.