mlsanityData to 7 Oct 2026

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

arXiv 2402.18216 ↗First posted 28 Feb 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

With the recent emergence of powerful instruction-tuned large language models (LLMs), various helpful conversational Artificial Intelligence (AI) systems have been deployed across many applications. When prompted by users, these AI systems successfully perform a wide range of tasks as part of a conversation. To provide some sort of memory and context, such approaches typically condition their output on the entire conversational history. Although this sensitivity to the conversational history can often lead to improved performance on subsequent tasks, we find that performance can in fact also be negatively impacted, if there is a task-switch. To the best of our knowledge, our work makes the first attempt to formalize the study of such vulnerabilities and interference of tasks in conversational LLMs caused by task-switches in the conversational history. Our experiments across 5 datasets with 15 task switches using popular LLMs reveal that many of the task-switches can lead to significant performance degradation.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

32 of 43 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  12. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
20 more
  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  2. 2023PaLM 2 Technical Reportcited by 1,154
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  4. 2022Transformers learn in-context by gradient descentcited by 714
  5. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  8. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  9. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 111
  10. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  11. 2023System 2 Attention (is something you might need too)cited by 85
  12. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  13. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  14. 2019TWEETQA: A Social Media Focused Question Answering Datasetcited by 55
  15. 2022Transformers generalize differently from information stored in context vs in weightscited by 49
  16. 2020Topic-Aware Multi-turn Dialogue Modelingcited by 46
  17. 2021SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systemscited by 34
  18. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attackscited by 30
  19. 2021TIAGE: A Benchmark for Topic-Shift Aware Dialog Modelingcited by 22
  20. 2023Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 9.

19 papers.

Sort:

By area:

  1. 2024Cognitive Overload Attack:Prompt Injection for Long Contextcited by 13
  2. 2024Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directionscited by 10
  3. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Surveycited by 10
  4. 2025OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Modelscited by 7
  5. 2026Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interactioncited by 7
  6. 2025ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Modelscited by 6
  7. 2025Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoningcited by 5
  8. 2026Mitigating Conversational Inertia in Multi-Turn Agentscited by 5
  9. 2025Vision-Speech Models: Teaching Speech Models to Converse about Imagescited by 4
  10. 2025Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue Historycited by 4
  11. 2026A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilitiescited by 4
  12. 2025Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agentscited by 3
7 more
  1. 2026APWA: A Distributed Architecture for Parallelizable Agentic Workflowscited by 1
  2. 2025When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agentscited by 0
  3. 2026Persuasion Propagation: Does Persuasion Change AI Agent Behavior?cited by 0
  4. 2026Retcon -- a Prompt-Based Technique for Precise Control of LLMs in Conversationscited by 0
  5. 2026Multimodal Task Interference: A Benchmark and Analysis of History-Target Mismatch in Multimodal LLMscited by 0
  6. 2026At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalizationcited by 0
  7. 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Historiescited by 0