mlsanityData to 7 Oct 2026

LLMs on Drugs: Language Models Are Few-Shot Consumers

arXiv 2512.18546 ↗First posted 21 Dec 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are sensitive to the personas imposed on them at inference time, yet prompt-level "drug" interventions have never been benchmarked rigorously. We present the first controlled study of psychoactive framings on GPT-5-mini using ARC-Challenge. Four single-sentence prompts -- LSD, cocaine, alcohol, and cannabis -- are compared against a sober control across 100 validation items per condition, with deterministic decoding, full logging, Wilson confidence intervals, and Fisher exact tests. Control accuracy is 0.45; alcohol collapses to 0.10 (p = 3.2e-8), cocaine to 0.21 (p = 4.9e-4), LSD to 0.19 (p = 1.3e-4), and cannabis to 0.30 (p = 0.041), largely because persona prompts disrupt the mandated "Answer: <LETTER>" template. Persona text therefore behaves like a "few-shot consumable" that can destroy reliability without touching model weights. All experimental code, raw results, and analysis scripts are available at https://github.com/lexdoudkin/llms-on-drugs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 34 references that are in the map. First posted on arXiv 21 Dec 2025.

Builds on

27 of 34 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  12. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
15 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  7. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  8. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  10. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  11. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  12. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  13. 2023Role-Play with Large Language Modelscited by 290
  14. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  15. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107

Built on by

Not cited yet.