# When Does a Laugh Begin? Structured Annotator Disagreement in Temporal Laughter Localization

arXiv: https://arxiv.org/abs/2609.06646 · HTML: https://mlsanity.com/paper/2609.06646/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 6 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Diagnosing Error in Temporal Action Detectors](https://mlsanity.com/paper/1807.10706/) (2018)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)](https://mlsanity.com/paper/2304.14803/) (2023)
- [Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline](https://mlsanity.com/paper/2303.12930/) (2023)
- [SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models](https://mlsanity.com/paper/2312.09818/) (2023)
- [Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video](https://mlsanity.com/paper/1703.09026/) (2017)
- [Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos](https://mlsanity.com/paper/2208.01954/) (2022)
- [Much Ado About Time: Exhaustive Annotation of Temporal Data](https://mlsanity.com/paper/1607.07429/) (2016)
- [Impact of annotation modality on label quality and model performance in the automatic assessment of laughter in-the-wild](https://mlsanity.com/paper/2211.00794/) (2022)
- [LEAD Dataset: How Can Labels for Sound Event Detection Vary Depending on Annotators?](https://mlsanity.com/paper/2410.09778/) (2024)
- [MTLLFM: Multimodal-Temporal Laughter Localization: UR-FUNNY-Temporal and SMILE-Temporal Benchmarks with an Adaptive Multimodal Fusion Model](https://mlsanity.com/paper/2605.25409/) (2026)
- [SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter](https://mlsanity.com/paper/2605.28084/) (2026)

## Abstract (arXiv)

Annotators routinely disagree on laughter boundaries and subtle chuckles, yet temporal laughter localization typically evaluates against a single reference annotation. We show that this disagreement is structured rather than random noise. Re-annotating the SMILE-Temporal benchmark (672 videos, 1,683 events) with 3-5 annotators per video (alpha = 0.757), we find systematic patterns: disagreement is 1.73 times larger at offsets than onsets, far more common for chuckles than full laughs (77% vs. 20%), and predictable from event attributes (AUC = 0.831). Evaluating against a single annotator breaks down under this structure: system scores shift by 0.246 F1 depending on the chosen ground truth, correctly ranking systems only 69.7% of the time (vs. 80% against all annotators). We propose a disagreement-calibrated evaluation that scores predictions against the full annotator distribution using conformally calibrated tolerance bands (wider at offsets, 0.727s, than onsets, 0.5s). The per-annotator annotations and analysis code are available at https://github.com/WSCSports/MTLLFM-temporal-laughter-localization.