# MMASD: A Multimodal Dataset for Autism Intervention Analysis

arXiv: https://arxiv.org/abs/2306.08243 · HTML: https://mlsanity.com/paper/2306.08243/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 12 of 12)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020)
- [Guided Weak Supervision for Action Recognition with Scarce Data to Assess Skills of Children with Autism](https://mlsanity.com/paper/1911.04140/) (2019)
- [Cross-Recurrence Quantification Analysis of Categorical and Continuous Time Series: an R package](https://mlsanity.com/paper/1310.0201/) (2013)
- [A Two-stage Multi-modal Affect Analysis Framework for Children with Autism Spectrum Disorder](https://mlsanity.com/paper/2106.09199/) (2021)
- [Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection](https://mlsanity.com/paper/2302.08293/) (2023)
- [Dyadic Movement Synchrony Estimation Under Privacy-preserving Conditions](https://mlsanity.com/paper/2208.01100/) (2022)
- [Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer](https://mlsanity.com/paper/2208.01161/) (2022)

## Built on by (most cited first, 8 of 8)

- [Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition](https://mlsanity.com/paper/2406.02554/) (2024)
- [MOOSE: Pay Attention to Temporal Dynamics for Video Understanding via Optical Flows](https://mlsanity.com/paper/2506.01119/) (2025)
- [OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization](https://mlsanity.com/paper/2602.10635/) (2026)
- [Unlocking Multi-Site Clinical Data: A Federated Approach to Privacy-First Child Autism Behavior Analysis](https://mlsanity.com/paper/2604.02616/) (2026)
- [Physics Augmented Tuple Transformer for Autism Severity Level Detection](https://mlsanity.com/paper/2409.18438/) (2024)
- [Improving Autism Detection with Multimodal Behavioral Analysis](https://mlsanity.com/paper/2509.21352/) (2025)
- [BuddyBench: A Privacy-Constrained Multi-Task Benchmark for Pediatric Social-Communication Personalization](https://mlsanity.com/paper/2605.28089/) (2026)
- [ChildGaze: A Benchmark Dataset for Collaborative Behavior Understanding in Children](https://mlsanity.com/paper/2609.06353/) (2026)

## Abstract (arXiv)

Autism spectrum disorder (ASD) is a developmental disorder characterized by significant social communication impairments and difficulties perceiving and presenting communication cues. Machine learning techniques have been broadly adopted to facilitate autism studies and assessments. However, computational models are primarily concentrated on specific analysis and validated on private datasets in the autism community, which limits comparisons across models due to privacy-preserving data sharing complications. This work presents a novel privacy-preserving open-source dataset, MMASD as a MultiModal ASD benchmark dataset, collected from play therapy interventions of children with Autism. MMASD includes data from 32 children with ASD, and 1,315 data samples segmented from over 100 hours of intervention recordings. To promote public access, each data sample consists of four privacy-preserving modalities of data; some of which are derived from original videos: (1) optical flow, (2) 2D skeleton, (3) 3D skeleton, and (4) clinician ASD evaluation scores of children, e.g., ADOS scores. MMASD aims to assist researchers and therapists in understanding children's cognitive status, monitoring their progress during therapy, and customizing the treatment plan accordingly. It also has inspiration for downstream tasks such as action quality assessment and interpersonal synchrony estimation. MMASD dataset can be easily accessed at https://github.com/Li-Jicheng/MMASD-A-Multimodal-Dataset-for-Autism-Intervention-Analysis.