# Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts

arXiv: https://arxiv.org/abs/2205.13884 · HTML: https://mlsanity.com/paper/2205.13884/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Internet-Augmented Dialogue Generation](https://mlsanity.com/paper/2107.07566/) (2021)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media](https://mlsanity.com/paper/1710.05429/) (2017)
- [ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs](https://mlsanity.com/paper/2112.07622/) (2021)

## Built on by (most cited first, 23 of 23)

- [Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safety](https://mlsanity.com/paper/2206.13349/) (2022)
- [Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety](https://mlsanity.com/paper/2312.06798/) (2023)
- [ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance](https://mlsanity.com/paper/2305.08010/) (2023)
- [WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions](https://mlsanity.com/paper/2406.12058/) (2024)
- [Process Knowledge-infused Learning for Clinician-friendly Explanations](https://mlsanity.com/paper/2306.09824/) (2023)
- [A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for Depression](https://mlsanity.com/paper/2311.13852/) (2023)
- [KSAT: Knowledge-infused Self Attention Transformer -- Integrating Multiple Domain-Specific Contexts](https://mlsanity.com/paper/2210.04307/) (2022)
- [Datasets for Depression Modeling in Social Media: An Overview](https://mlsanity.com/paper/2503.21513/) (2025)
- [Demo Alleviate: Demonstrating Artificial Intelligence Enabled Virtual Assistance for Telehealth: The Mental Health Case](https://mlsanity.com/paper/2304.00025/) (2023)
- [Knowledge-Infused Self Attention Transformers](https://mlsanity.com/paper/2306.13501/) (2023)
- [IERL: Interpretable Ensemble Representation Learning -- Combining CrowdSourced Knowledge and Distributed Semantic Representations](https://mlsanity.com/paper/2306.13865/) (2023)
- [Your Model Is Not Predicting Depression Well And That Is Why: A Case Study of PRIMATE Dataset](https://mlsanity.com/paper/2403.00438/) (2024)
- [Towards Explainable and Safe Conversational Agents for Mental Health: A Survey](https://mlsanity.com/paper/2304.13191/) (2023)
- [Deep Knowledge-Infusion For Explainable Depression Detection](https://mlsanity.com/paper/2409.02122/) (2024)
- [Bridging Information Gaps with Comprehensive Answers: Improving the Diversity and Informativeness of Follow-Up Questions](https://mlsanity.com/paper/2502.17715/) (2025)
- [Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models](https://mlsanity.com/paper/2308.12272/) (2023)
- [Navigating Healthcare Insights: A Birds Eye View of Explainability with Knowledge Graphs](https://mlsanity.com/paper/2309.16593/) (2023)
- [Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization](https://mlsanity.com/paper/2501.16106/) (2025)
- [Evaluating Lexicon Incorporation for Depression Symptom Estimation](https://mlsanity.com/paper/2404.19359/) (2024)
- [IoT-Based Preventive Mental Health Using Knowledge Graphs and Standards for Better Well-Being](https://mlsanity.com/paper/2406.13791/) (2024)
- [An Ubuntu-Guided Large Language Model Framework for Cognitive Behavioral Mental Health Dialogue](https://mlsanity.com/paper/2601.06875/) (2026)
- [SynSym: A Synthetic Data Generation Framework for Psychiatric Symptom Identification](https://mlsanity.com/paper/2603.21529/) (2026)
- [Structure vs. Chain-of-Thought: Evaluating LLM Criteria Extraction for Depression Severity](https://mlsanity.com/paper/2609.39049/) (2026)

## Abstract (arXiv)

Conversational Agents (CAs) powered with deep language models (DLMs) have shown tremendous promise in the domain of mental health. Prominently, the CAs have been used to provide informational or therapeutic services to patients. However, the utility of CAs to assist in mental health triaging has not been explored in the existing work as it requires a controlled generation of follow-up questions (FQs), which are often initiated and guided by the mental health professionals (MHPs) in clinical settings. In the context of depression, our experiments show that DLMs coupled with process knowledge in a mental health questionnaire generate 12.54% and 9.37% better FQs based on similarity and longest common subsequence matches to questions in the PHQ-9 dataset respectively, when compared with DLMs without process knowledge support. Despite coupling with process knowledge, we find that DLMs are still prone to hallucination, i.e., generating redundant, irrelevant, and unsafe FQs. We demonstrate the challenge of using existing datasets to train a DLM for generating FQs that adhere to clinical process knowledge. To address this limitation, we prepared an extended PHQ-9 based dataset, PRIMATE, in collaboration with MHPs. PRIMATE contains annotations regarding whether a particular question in the PHQ-9 dataset has already been answered in the user's initial description of the mental health condition. We used PRIMATE to train a DLM in a supervised setting to identify which of the PHQ-9 questions can be answered directly from the user's post and which ones would require more information from the user. Using performance analysis based on MCC scores, we show that PRIMATE is appropriate for identifying questions in PHQ-9 that could guide generative DLMs towards controlled FQ generation suitable for aiding triaging. Dataset created as a part of this research: https://github.com/primate-mh/Primate2022