# APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations

arXiv: https://arxiv.org/abs/2209.08207 · HTML: https://mlsanity.com/paper/2209.08207/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models](https://mlsanity.com/paper/2201.05337/) (2022)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.01161/) (2017)
- [Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer](https://mlsanity.com/paper/1805.07685/) (2018)
- [Structural Adapters in Pretrained Language Models for AMR-to-text Generation](https://mlsanity.com/paper/2103.09120/) (2021)
- [Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen](https://mlsanity.com/paper/2005.00701/) (2020)
- [A Deep Reinforcement Learning Chatbot](https://mlsanity.com/paper/1709.02349/) (2017)
- [Discourse-Aware Neural Rewards for Coherent Text Generation](https://mlsanity.com/paper/1805.03766/) (2018)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization](https://mlsanity.com/paper/2010.05090/) (2020)
- [How Do We Answer Complex Questions: Discourse Structure of Long-form Answers](https://mlsanity.com/paper/2203.11048/) (2022)
- [Contextual Text Style Transfer](https://mlsanity.com/paper/2005.00136/) (2020)
- [Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation](https://mlsanity.com/paper/2006.16823/) (2020)
- [The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error](https://mlsanity.com/paper/2203.11317/) (2022)
- [On Robustness and Bias Analysis of BERT-based Relation Extraction](https://mlsanity.com/paper/2009.06206/) (2020)

## Built on by (most cited first, 23 of 23)

- [Learning Interpretable Style Embeddings via Prompting LLMs](https://mlsanity.com/paper/2305.12696/) (2023)
- [DisCGen: A Framework for Discourse-Informed Counterspeech Generation](https://mlsanity.com/paper/2311.18147/) (2023)
- [MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages](https://mlsanity.com/paper/2404.02037/) (2024)
- [D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias](https://mlsanity.com/paper/2305.17013/) (2023)
- [Through the Lens of Core Competency: Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2308.07902/) (2023)
- [Multilingual and Explainable Text Detoxification with Parallel Corpora](https://mlsanity.com/paper/2412.11691/) (2024)
- [Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios](https://mlsanity.com/paper/2410.11114/) (2024)
- [Multilingual Content Moderation: A Case Study on Reddit](https://mlsanity.com/paper/2302.09618/) (2023)
- [Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewriting](https://mlsanity.com/paper/2305.14755/) (2023)
- [CMD: a framework for Context-aware Model self-Detoxification](https://mlsanity.com/paper/2308.08295/) (2023)
- [Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learning](https://mlsanity.com/paper/2310.10707/) (2023)
- [An Active Learning Framework for Inclusive Generation by Large Language Models](https://mlsanity.com/paper/2410.13641/) (2024)
- [Steering Large Language Models with Register Analysis for Arbitrary Style Transfer](https://mlsanity.com/paper/2505.00679/) (2025)
- [SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators](https://mlsanity.com/paper/2502.06394/) (2025)
- [MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations](https://mlsanity.com/paper/2305.19981/) (2023)
- [Subtle Misogyny Detection and Mitigation: An Expert-Annotated Dataset](https://mlsanity.com/paper/2311.09443/) (2023)
- [Promoting Constructive Deliberation: Reframing for Receptiveness](https://mlsanity.com/paper/2405.15067/) (2024)
- [GPT-DETOX: An In-Context Learning-Based Paraphraser for Text Detoxification](https://mlsanity.com/paper/2404.03052/) (2024)
- [BanglaNirTox: A Large-scale Parallel Corpus for Explainable AI in Bengali Text Detoxification](https://mlsanity.com/paper/2511.01512/) (2025)
- [Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community](https://mlsanity.com/paper/2404.01158/) (2024)
- [Explainable Detection of Implicit Influential Patterns in Conversations via Data Augmentation](https://mlsanity.com/paper/2506.14211/) (2025)
- [A Survey of Toxicity Detection and Mitigation Strategies for Multilingual Language Models](https://mlsanity.com/paper/2606.25380/) (2026)
- [From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation](https://mlsanity.com/paper/2608.25605/) (2026)

## Abstract (arXiv)

Using style-transfer models to reduce offensiveness of social media comments can help foster a more inclusive environment. However, there are no sizable datasets that contain offensive texts and their inoffensive counterparts, and fine-tuning pretrained models with limited labeled data can lead to the loss of original meaning in the style-transferred text. To address this issue, we provide two major contributions. First, we release the first publicly-available, parallel corpus of offensive Reddit comments and their style-transferred counterparts annotated by expert sociolinguists. Then, we introduce the first discourse-aware style-transfer models that can effectively reduce offensiveness in Reddit text while preserving the meaning of the original text. These models are the first to examine inferential links between the comment and the text it is replying to when transferring the style of offensive Reddit text. We propose two different methods of integrating discourse relations with pretrained transformer models and evaluate them on our dataset of offensive comments from Reddit and their inoffensive counterparts. Improvements over the baseline with respect to both automatic metrics and human evaluation indicate that our discourse-aware models are better at preserving meaning in style-transferred text when compared to the state-of-the-art discourse-agnostic models.