# A Multi-Agent Framework for Mitigating Dialect Biases in Privacy Policy Question-Answering Systems

arXiv: https://arxiv.org/abs/2506.02998 · HTML: https://mlsanity.com/paper/2506.02998/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 17 of 17)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [AutoAgents: A Framework for Automatic Agent Generation](https://mlsanity.com/paper/2309.17288/) (2023)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings](https://mlsanity.com/paper/1904.01596/) (2019)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation](https://mlsanity.com/paper/2402.10178/) (2024)
- [DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages](https://mlsanity.com/paper/2403.11009/) (2024)
- [PolicyQA: A Reading Comprehension Dataset for Privacy Policies](https://mlsanity.com/paper/2010.02557/) (2020)
- [Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset](https://mlsanity.com/paper/2008.09159/) (2020)
- [Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies](https://mlsanity.com/paper/2004.11131/) (2020)
- [Intent Classification and Slot Filling for Privacy Policies](https://mlsanity.com/paper/2101.00123/) (2021)
- [TADA: Task-Agnostic Dialect Adapters for English](https://mlsanity.com/paper/2305.16651/) (2023)
- [DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules](https://mlsanity.com/paper/2305.13406/) (2023)
- [Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systems](https://mlsanity.com/paper/2406.14545/) (2024)
- [PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English](https://mlsanity.com/paper/2212.10011/) (2022)

## Built on by (most cited first, 2 of 2)

- [Examining Agents' Bias Amplification versus Suppression in Multi-Agent Systems](https://mlsanity.com/paper/2605.28098/) (2026)
- [Retrieval Sensitivity to Identity Signals in Queries](https://mlsanity.com/paper/2609.36534/) (2026)

## Abstract (arXiv)

Privacy policies inform users about data collection and usage, yet their complexity limits accessibility for diverse populations. Existing Privacy Policy Question Answering (QA) systems exhibit performance disparities across English dialects, disadvantaging speakers of non-standard varieties. We propose a novel multi-agent framework inspired by human-centered design principles to mitigate dialectal biases. Our approach integrates a Dialect Agent, which translates queries into Standard American English (SAE) while preserving dialectal intent, and a Privacy Policy Agent, which refines predictions using domain expertise. Unlike prior approaches, our method does not require retraining or dialect-specific fine-tuning, making it broadly applicable across models and domains. Evaluated on PrivacyQA and PolicyQA, our framework improves GPT-4o-mini's zero-shot accuracy from 0.394 to 0.601 on PrivacyQA and from 0.352 to 0.464 on PolicyQA, surpassing or matching few-shot baselines without additional training data. These results highlight the effectiveness of structured agent collaboration in mitigating dialect biases and underscore the importance of designing NLP systems that account for linguistic diversity to ensure equitable access to privacy information.