Self-Evolving Human-Centered Framework for Explainable Depression Symptom Annotation
On the map: Synthetic data and self-improvement for LLMs → Medical synthetic data and agents
Abstract (arXiv)
Annotation quality is a major bottleneck in building reliable and explainable artificial intelligence (XAI) systems for mental health research. In depression-related datasets, labels are often assigned without structured evidence, symptom-level justification, or traceable alignment with the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR), limiting both transparency and downstream model interpretability. We propose a self-evolving, expert-in-the-loop annotation framework for Major Depressive Disorder (MDD) that combines large language model (LLM)-assisted labeling with expert verification. The framework is intended to support the construction of explainable, DSM-5-TR-aligned datasets rather than to perform clinical diagnosis. It operates in three stages: candidate evidence selection from textual records, criterion-level DSM-5-TR analysis, and case-level synthesis that produces label-level diagnostic and severity annotations. A dual-memory architecture, composed of Example Memory and Reflection Memory, is designed to internalize expert feedback and iteratively improve future annotations without retraining. We describe this mechanism and leave its evaluation across multiple feedback cycles to future work. In addition to final labels, the framework exports clinical evidence, reasoning traces, and edit histories, enabling comprehensive auditability. In a pilot study using expert-reviewed samples, the proposed approach improves annotation consistency and explainability while reducing manual revision effort.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 9 of its 30 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
9 of 30 references.
References · 9
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
- 2024MEGAnno+: A Human-LLM Collaborative Annotation Systemcited by 47
- 2022Symptom Identification for Interpretable Detection of Multiple Mental Disorderscited by 20
- 2025Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretationcited by 6
- 2025ReDSM5: A Reddit Dataset for DSM-5 Depression Detectioncited by 5
- 2025A Gold Standard Dataset and Evaluation Framework for Depression Detection and Explanation in Social Media using LLMscited by 4
- 2024Using LLMs to Aid Annotation and Collection of Clinically-Enriched Data in Bipolar Disorder and Schizophreniacited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Interpretable Symptom Vectors for Depression in a Large Language Model
- 2026SynSym: A Synthetic Data Generation Framework for Psychiatric Symptom Identification
- 2023From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models
- 2026ADAPTS: Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms
- 2026Dep-LLM: Training-Free Depression Diagnosis via Evidence-Guided Structured Multi-factor with Reliable LLM Reasoning
- 2024InterMind: Doctor-Patient-Family Interactive Depression Assessment Empowered by Large Language Models
- 2025Enhancing Depression Detection with Chain-of-Thought Prompting: From Emotion to Reasoning Using Large Language Models
- 2025Systematic Evaluation of Machine-Generated Reasoning and PHQ-9 Labeling for Depression Detection Using Large Language Models