# Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts

arXiv: https://arxiv.org/abs/2212.01700 · HTML: https://mlsanity.com/paper/2212.01700/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 3 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate](https://mlsanity.com/paper/1809.02208/) (2018)
- [A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation](https://mlsanity.com/paper/2001.05139/) (2020)
- [Towards Controllable Biases in Language Generation](https://mlsanity.com/paper/2005.00268/) (2020)
- [Ethical Challenges in Data-Driven Dialogue Systems](https://mlsanity.com/paper/1711.09050/) (2017)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [Examining Gender Bias in Languages with Grammatical Gender](https://mlsanity.com/paper/1909.02224/) (2019)
- [Revealing Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2104.08728/) (2021)
- [Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications](https://mlsanity.com/paper/2205.06828/) (2022)
- [Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View](https://mlsanity.com/paper/2203.12258/) (2022)
- [Defining and Evaluating Fair Natural Language Generation](https://mlsanity.com/paper/2008.01548/) (2020)
- [Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation](https://mlsanity.com/paper/2205.09830/) (2022)

## Built on by (most cited first, 7 of 7)

- [The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text](https://mlsanity.com/paper/2311.09807/) (2023)
- [Evaluating Large Language Models on Controlled Generation Tasks](https://mlsanity.com/paper/2310.14542/) (2023)
- [Benchmarking Linguistic Diversity of Large Language Models](https://mlsanity.com/paper/2412.10271/) (2024)
- [Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluations](https://mlsanity.com/paper/2503.06987/) (2025)
- [Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach](https://mlsanity.com/paper/2310.06680/) (2023)
- [Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT](https://mlsanity.com/paper/2405.06996/) (2024)
- [Limited Linguistic Diversity in Embodied AI Datasets](https://mlsanity.com/paper/2601.03136/) (2026)

## Abstract (arXiv)

We present a robust methodology for evaluating biases in natural language generation(NLG) systems. Previous works use fixed hand-crafted prefix templates with mentions of various demographic groups to prompt models to generate continuations for bias analysis. These fixed prefix templates could themselves be specific in terms of styles or linguistic structures, which may lead to unreliable fairness conclusions that are not representative of the general trends from tone varying prompts. To study this problem, we paraphrase the prompts with different syntactic structures and use these to evaluate demographic bias in NLG systems. Our results suggest similar overall bias trends but some syntactic structures lead to contradictory conclusions compared to past works. We show that our methodology is more robust and that some syntactic structures prompt more toxic content while others could prompt less biased generation. This suggests the importance of not relying on a fixed syntactic structure and using tone-invariant prompts. Introducing syntactically-diverse prompts can achieve more robust NLG (bias) evaluation.