# D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias

arXiv: https://arxiv.org/abs/2305.17013 · HTML: https://mlsanity.com/paper/2305.17013/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Active Learning by Acquiring Contrastive Examples](https://mlsanity.com/paper/2109.03764/) (2021)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech](https://mlsanity.com/paper/2107.08720/) (2021)
- [On Statistical Bias In Active Learning: How and When To Fix It](https://mlsanity.com/paper/2101.11665/) (2021)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision](https://mlsanity.com/paper/2003.00393/) (2020)
- [On the Importance of Effectively Adapting Pretrained Language Models for Active Learning](https://mlsanity.com/paper/2104.08320/) (2021)
- [APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations](https://mlsanity.com/paper/2209.08207/) (2022)
- [Multi-task Active Learning for Pre-trained Transformer-based Models](https://mlsanity.com/paper/2208.05379/) (2022)
- [Reducing Confusion in Active Learning for Part-Of-Speech Tagging](https://mlsanity.com/paper/2011.00767/) (2020)
- [On Robustness and Bias Analysis of BERT-based Relation Extraction](https://mlsanity.com/paper/2009.06206/) (2020)
- [Multilingual Content Moderation: A Case Study on Reddit](https://mlsanity.com/paper/2302.09618/) (2023)
- [Classifier Risk Estimation under Limited Labeling Resources](https://mlsanity.com/paper/1607.02665/) (2016)
- [Pre-trained Language Model Based Active Learning for Sentence Matching](https://mlsanity.com/paper/2010.05522/) (2020)
- [Cross-lingual Emotion Detection](https://mlsanity.com/paper/2106.06017/) (2021)
- [Mitigating Sampling Bias and Improving Robustness in Active Learning](https://mlsanity.com/paper/2109.06321/) (2021)

## Built on by (most cited first, 12 of 12)

- [DisCGen: A Framework for Discourse-Informed Counterspeech Generation](https://mlsanity.com/paper/2311.18147/) (2023)
- [Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios](https://mlsanity.com/paper/2410.11114/) (2024)
- [Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agents](https://mlsanity.com/paper/2410.14141/) (2024)
- [Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI](https://mlsanity.com/paper/2501.06129/) (2025)
- [An Active Learning Framework for Inclusive Generation by Large Language Models](https://mlsanity.com/paper/2410.13641/) (2024)
- [Scoping Review of Active Learning Strategies and their Evaluation Environments for Entity Recognition Tasks](https://mlsanity.com/paper/2407.03895/) (2024)
- [Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community](https://mlsanity.com/paper/2404.01158/) (2024)
- [Enhancing BERT Fine-Tuning for Sentiment Analysis in Lower-Resourced Languages](https://mlsanity.com/paper/2512.01460/) (2025)
- [Moonworks Lunara Aesthetic Dataset](https://mlsanity.com/paper/2601.07941/) (2026)
- [Moonworks Lunara Aesthetic II: An Image Variation Dataset](https://mlsanity.com/paper/2602.01666/) (2026)
- [QUORUM: QUality-Optimized Routing Using Multiple annotators](https://mlsanity.com/paper/2608.27974/) (2026)
- [Moonworks Lunara: Modeling Artistic Intelligence](https://mlsanity.com/paper/2609.22272/) (2026)

## Abstract (arXiv)

Despite recent advancements, NLP models continue to be vulnerable to bias. This bias often originates from the uneven distribution of real-world data and can propagate through the annotation process. Escalated integration of these models in our lives calls for methods to mitigate bias without overbearing annotation costs. While active learning (AL) has shown promise in training models with a small amount of annotated data, AL's reliance on the model's behavior for selective sampling can lead to an accumulation of unwanted bias rather than bias mitigation. However, infusing clustering with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL's annotation efficiency. In this paper, we propose a novel adaptive clustering-based active learning algorithm, D-CALM, that dynamically adjusts clustering and annotation efforts in response to an estimated classifier error-rate. Experiments on eight datasets for a diverse set of text classification tasks, including emotion, hatespeech, dialog act, and book type detection, demonstrate that our proposed algorithm significantly outperforms baseline AL approaches with both pretrained transformers and traditional Support Vector Machines. D-CALM showcases robustness against different measures of information gain and, as evident from our analysis of label and error distribution, can significantly reduce unwanted model bias.