# How Much User Context Do We Need? Privacy by Design in Mental Health NLP Application

arXiv: https://arxiv.org/abs/2209.02022 · HTML: https://mlsanity.com/paper/2209.02022/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 84 references that are in the map.
- First posted on arXiv 5 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds](https://mlsanity.com/paper/1405.7085/) (2014)
- [Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?](https://mlsanity.com/paper/2104.07762/) (2021)
- [On the Privacy Risks of Algorithmic Fairness](https://mlsanity.com/paper/2011.03731/) (2020)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy](https://mlsanity.com/paper/2009.06389/) (2020)
- [Privacy-preserving Neural Representations of Text](https://mlsanity.com/paper/1808.09408/) (2018)
- [Generalised Differential Privacy for Text Document Processing](https://mlsanity.com/paper/1811.10256/) (2018)
- [Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately](https://mlsanity.com/paper/2012.04104/) (2020)
- [Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12](https://mlsanity.com/paper/1709.02753/) (2017)
- [Three Tools for Practical Differential Privacy](https://mlsanity.com/paper/1812.02890/) (2018)
- [Operationalizing the Legal Principle of Data Minimization for Personalization](https://mlsanity.com/paper/2005.13718/) (2020)
- [I Am Not What I Write: Privacy Preserving Text Representation Learning](https://mlsanity.com/paper/1907.03189/) (2019)
- [Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records](https://mlsanity.com/paper/1906.05000/) (2019)
- [Differential Data Analysis for Recommender Systems](https://mlsanity.com/paper/1310.0894/) (2013)
- [Differential Privacy: An Estimation Theory-Based Method for Choosing Epsilon](https://mlsanity.com/paper/1510.00917/) (2015)
- [Contextual Consent: Ethical Mining of Social Media for Health Research](https://mlsanity.com/paper/1701.07765/) (2017)
- [Living Labs - An Ethical Challenge for Researchers and Platform Providers](https://mlsanity.com/paper/1706.07202/) (2017)

## Built on by (most cited first, 2 of 2)

- [FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health](https://mlsanity.com/paper/2509.14275/) (2025)
- [Cloaked Classifiers: Pseudonymization Strategies on Sensitive Classification Tasks](https://mlsanity.com/paper/2406.17875/) (2024)

## Abstract (arXiv)

Clinical NLP tasks such as mental health assessment from text, must take social constraints into account - the performance maximization must be constrained by the utmost importance of guaranteeing privacy of user data. Consumer protection regulations, such as GDPR, generally handle privacy by restricting data availability, such as requiring to limit user data to 'what is necessary' for a given purpose. In this work, we reason that providing stricter formal privacy guarantees, while increasing the volume of user data in the model, in most cases increases benefit for all parties involved, especially for the user. We demonstrate our arguments on two existing suicide risk assessment datasets of Twitter and Reddit posts. We present the first analysis juxtaposing user history length and differential privacy budgets and elaborate how modeling additional user context enables utility preservation while maintaining acceptable user privacy guarantees.