mlsanityData to 14 Oct 2026

Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models

arXiv 2208.07852 ↗First posted 16 Aug 2022cs.CL cs.HC cs.LGcited by 70

Not on the map. Why?

Abstract (arXiv)

State-of-the-art neural language models can now be used to solve ad-hoc language tasks through zero-shot prompting without the need for supervised training. This approach has gained popularity in recent years, and researchers have demonstrated prompts that achieve strong accuracy on specific NLP tasks. However, finding a prompt for new tasks requires experimentation. Different prompt templates with different wording choices lead to significant accuracy differences. PromptIDE allows users to experiment with prompt variations, visualize prompt performance, and iteratively optimize prompts. We developed a workflow that allows users to first focus on model feedback using small data before moving on to a large data regime that allows empirical grounding of promising prompts using quantitative measures of the task. The tool then allows easy deployment of the newly created ad-hoc models. We demonstrate the utility of PromptIDE (demo at http://prompt.vizhub.ai) and our workflow using several real-world use cases.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 16 Aug 2022.

Builds on

31 of 46 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
19 more
  1. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  2. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  3. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  4. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 496
  5. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  6. 2021True Few-Shot Learning with Language Modelscited by 344
  7. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  8. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  9. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  10. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 252
  11. 2021How Many Data Points is a Prompt Worth?cited by 198
  12. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  13. 2020Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Modelscited by 49
  14. 2017Understanding Hidden Memories of Recurrent Neural Networkscited by 44
  15. 2021Dodrio: Exploring Transformer Models with Interactive Visualizationcited by 28
  16. 2021VisQA: X-raying Vision and Language Reasoning in Transformerscited by 19
  17. 2021LMdiff: A Visual Diff Tool to Compare Language Modelscited by 17
  18. 2021T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLPcited by 11
  19. 2018Progressive Data Science: Potential and Challengescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 30; 2024: 19; 2025: 14; 2026: 4.

70 papers.

Sort:

By area:

  1. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  2. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
  3. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
  4. 2023Generative AIcited by 120
  5. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Costcited by 109
  6. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  7. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  8. 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 83
  9. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 74
  10. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 65
  11. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 62
  12. 2023PromptMagician: Interactive Prompt Engineering for Text-to-Image Creationcited by 58
38 more
  1. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
  2. 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
  3. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thoughtcited by 41
  4. 2023Visual cognition in multimodal large language modelscited by 38
  5. 2023Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Promptingcited by 36
  6. 2023PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactionscited by 34
  7. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  8. 2023Foundation Models Meet Visualizations: Challenges and Opportunitiescited by 31
  9. 2024LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Modelscited by 29
  10. 2024The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiringcited by 29
  11. 2023Visual Analytics For Machine Learning: A Data Perspective Surveycited by 25
  12. 2024LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systemscited by 21
  13. 2023Tree Prompting: Efficient Task Adaptation without Fine-Tuningcited by 18
  14. 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applicationscited by 17
  15. 2024From Prompt Engineering to Prompt Science With Human in the Loopcited by 12
  16. 2024Exploring Prompt Engineering Practices in the Enterprisecited by 11
  17. 2024Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Productscited by 11
  18. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  19. 2023Can ChatGPT Pass An Introductory Level Functional Language Programming Course?cited by 9
  20. 2024Wordflow: Social Prompt Engineering for Large Language Modelscited by 9
  21. 2023Towards an Understanding and Explanation for Mixed-Initiative Artificial Scientific Text Detectioncited by 8
  22. 2023Segment Anything Meets Semantic Communicationcited by 8
  23. 2023PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflowscited by 8
  24. 2024How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments?cited by 8
  25. 2025DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoningcited by 8
  26. 2024KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Promptscited by 7
  27. 2024JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Modelscited by 7
  28. 2024ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion Mapcited by 7
  29. 2025An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniquescited by 7
  30. 2023A Case Study in Engineering a Conversational Programming Assistant's Personacited by 6
  31. 2023LMCanvas: Object-Oriented Interaction to Personalize Large Language Model-Powered Writing Environmentscited by 6
  32. 2024Talaria: Interactively Optimizing Machine Learning Models for Efficient Inferencecited by 6
  33. 2025Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extractioncited by 6
  34. 2024Exploring Prompt Engineering: A Systematic Review with SWOT Analysiscited by 5
  35. 2026From Words to Widgets for Controllable LLM Generationcited by 5
  36. 2023Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Educationcited by 4
  37. 2024It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehensioncited by 4
  38. 2025LLM-Agents Driven Automated Simulation Testing and Analysis of small Uncrewed Aerial Systemscited by 4

50 most cited of 70 All 70 (JSON)