mlsanityData to 7 Oct 2026

Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

arXiv 2203.08979 ↗First posted 16 Mar 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English--Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 28 of its 54 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

28 of 54 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  7. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  8. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  9. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  10. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  11. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  12. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
16 more
  1. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  2. 2016A Persona-Based Neural Conversation Modelcited by 221
  3. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  4. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
  5. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  6. 2018Training Millions of Personalized Dialogue Agentscited by 126
  7. 2019Personalizing Dialogue Agents via Meta-Learningcited by 83
  8. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  9. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  10. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  11. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44
  12. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  13. 2019From English to Code-Switching: Transfer Learning with Strong Morphological Cluescited by 18
  14. 2018Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Trainingcited by 14
  15. 2019Topics to Avoid: Demoting Latent Confounds in Text Classificationcited by 14
  16. 2021Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switchingcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2; 2024: 0; 2025: 1.

6 papers.

  1. 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
  2. 2022KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understandingcited by 9
  3. 2023GlobalBench: A Benchmark for Global Progress in Natural Language Processingcited by 8
  4. 2023Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attentioncited by 4
  5. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  6. 2022Mixed-effects transformers for hierarchical adaptationcited by 2