# Interpreting Depression From Question-wise Long-term Video Recording of SDS Evaluation

arXiv: https://arxiv.org/abs/2106.13393 · HTML: https://mlsanity.com/paper/2106.13393/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 77 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [How to Construct Deep Recurrent Neural Networks](https://mlsanity.com/paper/1312.6026/) (2013)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications](https://mlsanity.com/paper/1606.03237/) (2016)
- [FaceBoxes: A CPU Real-time Face Detector with High Accuracy](https://mlsanity.com/paper/1708.05234/) (2017)
- [Revisiting Temporal Modeling for Video-based Person ReID](https://mlsanity.com/paper/1805.02104/) (2018)
- [Measuring Depression Symptom Severity from Spoken Language and 3D Facial Expressions](https://mlsanity.com/paper/1811.08592/) (2018)
- [Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis](https://mlsanity.com/paper/2101.00318/) (2021)
- [DEPA: Self-Supervised Audio Embedding for Depression Detection](https://mlsanity.com/paper/1910.13028/) (2019)
- [Energy-constrained Self-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2101.00316/) (2021)
- [AudVowelConsNet: A Phoneme-Level Based Deep CNN Architecture for Clinical Depression Diagnosis](https://mlsanity.com/paper/2010.16201/) (2020)
- [Dependency-aware Attention Control for Unconstrained Face Recognition with Image Sets](https://mlsanity.com/paper/1907.03030/) (2019)
- [Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling](https://mlsanity.com/paper/1806.00681/) (2018)
- [Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition](https://mlsanity.com/paper/1908.01174/) (2019)
- [Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video](https://mlsanity.com/paper/2010.10637/) (2020)
- [Identity-aware Facial Expression Recognition in Compressed Video](https://mlsanity.com/paper/2101.00317/) (2021)
- [Disentanglement for Discriminative Visual Recognition](https://mlsanity.com/paper/2006.07810/) (2020)

## Built on by (most cited first, 4 of 4)

- [Multi-modal Depression Estimation based on Sub-attentional Fusion](https://mlsanity.com/paper/2207.06180/) (2022)
- [Automatic Depression Assessment using Machine Learning: A Comprehensive Survey](https://mlsanity.com/paper/2506.18915/) (2025)
- [Deep 3D-CNN for Depression Diagnosis with Facial Video Recording of Self-Rating Depression Scale Questionnaire](https://mlsanity.com/paper/2107.10712/) (2021)
- [A Deep Learning approach for Depressive Symptoms assessment in Parkinson's disease patients using facial videos](https://mlsanity.com/paper/2505.03845/) (2025)

## Abstract (arXiv)

Self-Rating Depression Scale (SDS) questionnaire has frequently been used for efficient depression preliminary screening. However, the uncontrollable self-administered measure can be easily affected by insouciantly or deceptively answering, and producing the different results with the clinician-administered Hamilton Depression Rating Scale (HDRS) and the final diagnosis. Clinically, facial expression (FE) and actions play a vital role in clinician-administered evaluation, while FE and action are underexplored for self-administered evaluations. In this work, we collect a novel dataset of 200 subjects to evidence the validity of self-rating questionnaires with their corresponding question-wise video recording. To automatically interpret depression from the SDS evaluation and the paired video, we propose an end-to-end hierarchical framework for the long-term variable-length video, which is also conditioned on the questionnaire results and the answering time. Specifically, we resort to a hierarchical model which utilizes a 3D CNN for local temporal pattern exploration and a redundancy-aware self-attention (RAS) scheme for question-wise global feature aggregation. Targeting for the redundant long-term FE video processing, our RAS is able to effectively exploit the correlations of each video clip within a question set to emphasize the discriminative information and eliminate the redundancy based on feature pair-wise affinity. Then, the question-wise video feature is concatenated with the questionnaire scores for final depression detection. Our thorough evaluations also show the validity of fusing SDS evaluation and its video recording, and the superiority of our framework to the conventional state-of-the-art temporal modeling methods.