mlsanityData to 7 Oct 2026

Improving Formality Style Transfer with Context-Aware Rule Injection

arXiv 2106.00210 ↗First posted 1 Jun 2021cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Models pre-trained on large-scale regular text corpora often do not work well for user-generated data where the language styles differ significantly from the mainstream text. Here we present Context-Aware Rule Injection (CARI), an innovative method for formality style transfer (FST). CARI injects multiple rules into an end-to-end BERT-based encoder and decoder model. It learns to select optimal rules based on context. The intrinsic evaluation showed that CARI achieved the new highest performance on the FST benchmark dataset. Our extrinsic evaluation showed that CARI can greatly improve the regular pre-trained models' performance on several tweet sentiment analysis tasks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

23 of 46 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  9. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  10. 2017Toward Controlled Generation of Textcited by 271
  11. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  12. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
11 more
  1. 2017Style Transfer in Text: Exploration and Evaluationcited by 145
  2. 2019Generating Sentences from Disentangled Syntactic and Semantic Spacescited by 59
  3. 2017Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Modelscited by 57
  4. 2019Low-Memory Neural Network Training: A Technical Reportcited by 56
  5. 2019Controllable Paraphrase Generation with a Syntactic Exemplarcited by 44
  6. 2020Parallel Data Augmentation for Formality Style Transfercited by 39
  7. 2018Multi-Task Neural Models for Translating Between Styles Within and Across Languagescited by 28
  8. 2019Formality Style Transfer with Hybrid Textual Annotationscited by 16
  9. 2020Zero-shot Entity Linking with Efficient Long Range Sequence Modelingcited by 13
  10. 2017MoNoise: Modeling Noise Using a Modular Normalization Systemcited by 8
  11. 2018SeerNet at SemEval-2018 Task 1: Domain Adaptation for Affect in Tweetscited by 7

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 4; 2024: 2; 2025: 1.

13 papers.

Sort:

  1. 2023NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notescited by 48
  2. 2023Editing Personality for Large Language Modelscited by 22
  3. 2022Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Contextcited by 15
  4. 2022Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probingcited by 15
  5. 2023UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?cited by 14
  6. 2022Semi-Supervised Formality Style Transfer with Consistency Trainingcited by 12
  7. 2024ReadCtrl: Personalizing text generation with readability-controlled instruction learningcited by 10
  8. 2022Automated Identification of Eviction Status from Electronic Health Record Notescited by 6
  9. 2021From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Surveycited by 5
  10. 2025MedReadCtrl: Personalizing medical text generation with readability-controlled instruction learningcited by 5
  11. 2022Exploring Cross-lingual Textual Style Transfer with Large Multilingual Language Modelscited by 3
  12. 2023In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languagescited by 2
1 more
  1. 2024UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration of Prompt Engineering with GPT-4V for Dermatological Diagnosiscited by 0