mlsanityData to 7 Oct 2026

Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning

arXiv 2110.01643 ↗First posted 4 Oct 2021cs.CL cs.CRcited by 18

Not on the map. Why?

Abstract (arXiv)

Privacy is important considering the financial Domain as such data is highly confidential and sensitive. Natural Language Processing (NLP) techniques can be applied for text classification and entity detection purposes in financial domains such as customer feedback sentiment analysis, invoice entity detection, categorisation of financial documents by type etc. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training large models with such data. In this work, we propose a contextualized transformer (BERT and RoBERTa) based text classification model integrated with privacy features such as Differential Privacy (DP) and Federated Learning (FL). We present how to privately train NLP models and desirable privacy-utility tradeoffs and evaluate them on the Financial Phrase Bank dataset.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 4 Oct 2021.

Builds on

19 of 36 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2019Federated Machine Learning: Concept and Applicationscited by 409
  7. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  8. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  9. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  10. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  11. 2019Differentially Private Model Publishing for Deep Learningcited by 69
  12. 2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 62
7 more
  1. 2018Generalised Differential Privacy for Text Document Processingcited by 52
  2. 2021Federated Learning Meets Natural Language Processing: A Surveycited by 42
  3. 2020A Survey on Transfer Learning in Natural Language Processingcited by 33
  4. 2021Benchmarking Differential Privacy and Federated Learning for BERT Modelscited by 28
  5. 2020A survey on natural language processing (nlp) and applications in insurancecited by 3
  6. 2021FedPandemic: A Cross-Device Federated Learning Approach Towards Elementary Prognosis of Diseases During a Pandemiccited by 3
  7. 2020Differential Privacy and Natural Language Processing to Generate Contextually Similar Decoy Messages in Honey Encryption Schemecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 4; 2025: 7; 2026: 1.

18 papers.

Sort:

By area:

  1. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  2. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 74
  3. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 55
  4. 2022Differential Privacy in Natural Language Processing: The Story So Farcited by 12
  5. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  6. 2023DP-NMT: Scalable Differentially-Private Machine Translationcited by 7
  7. 2025A Survey on Current Trends and Recent Advances in Text Anonymizationcited by 5
  8. 2025Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunitiescited by 3
  9. 2023PrIeD-KIE: Towards Privacy Preserved Document Key Information Extractioncited by 2
  10. 2025Differentially Private Iterative Screening Rules for Linear Regressioncited by 2
  11. 2024FLMarket: Enabling Privacy-preserved Pre-training Data Pricing for Federated Learningcited by 1
  12. 2025The Transition from Centralized Machine Learning to Federated Learning for Mental Health in Education: A Survey of Current Methods and Future Directionscited by 1
6 more
  1. 2025Enhancing Privacy in the Early Detection of Sexual Predators Through Federated Learning and Differential Privacycited by 1
  2. 2025DP-DocLDM: Differentially Private Document Image Generation using Latent Diffusion Modelscited by 1
  3. 2025Optimizing Privacy-Preserving Primitives to Support LLM-Scale Applicationscited by 1
  4. 2024FLAIN: Mitigating Backdoor Attacks in Federated Learning via Flipping Weight Updates of Low-Activation Input Neuronscited by 0
  5. 2024Which PPML Would a User Choose? A Structured Decision Support Framework for Developers to Rank PPML Techniques Based on User Acceptance Criteriacited by 0
  6. 2026On the Impact of Anonymization on the Performance of Large Language Modelscited by 0