mlsanityData to 7 Oct 2026

AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts

arXiv 2110.01691 ↗First posted 4 Oct 2021cs.HC cs.CLcited by 265

Not on the map. Why?

Cited by 53 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although large language models (LLMs) have demonstrated impressive potential on simple tasks, their breadth of scope, lack of transparency, and insufficient controllability can make them less effective when assisting humans on more complex tasks. In response, we introduce the concept of Chaining LLM steps together, where the output of one step becomes the input for the next, thus aggregating the gains per step. We first define a set of LLM primitive operations useful for Chain construction, then present an interactive system where users can modify these Chains, along with their intermediate results, in a modular way. In a 20-person user study, we found that Chaining not only improved the quality of task outcomes, but also significantly enhanced system transparency, controllability, and sense of collaboration. Additionally, we saw that users developed new ways of interacting with LLMs through Chains: they leveraged sub-tasks to calibrate model expectations, compared and contrasted alternative strategies by observing parallel downstream effects, and debugged unexpected model outputs by "unit-testing" sub-components of a Chain. In two case studies, we further explore how LLM Chains may be used in future applications

Key facts. Cited by 265 papers in the map (data through 7 Oct 2026). Builds on 32 of its 104 references that are in the map. First posted on arXiv 4 Oct 2021.

Builds on

32 of 104 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  10. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  11. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  12. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
20 more
  1. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  2. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  3. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  4. 2016Exploring the Limits of Language Modelingcited by 172
  5. 2019Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Makingcited by 109
  6. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  7. 2021Detecting Hate Speech with GPT-3cited by 55
  8. 2020Consistency of a Recurrent Language Model With Respect to Incomplete Decodingcited by 54
  9. 2021What Context Features Can Transformer Language Models Use?cited by 49
  10. 2019Similar Image Search for Histopathology: SMILYcited by 41
  11. 2021Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiativecited by 28
  12. 2021GPT3-to-plan: Extracting plans from text using GPT-3cited by 26
  13. 2020AI Song Contest: Human-AI Co-Creation in Songwritingcited by 25
  14. 2021VizLinter: A Linter and Fixer Framework for Data Visualizationcited by 20
  15. 2016On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systemscited by 18
  16. 2021Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoningcited by 17
  17. 2021Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2cited by 15
  18. 2016Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revisioncited by 13
  19. 2021Nine Potential Pitfalls when Designing Human-AI Co-Creative Systemscited by 12
  20. 2021Understanding Emails and Drafting Responses -- An Approach Using GPT-3cited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 21; 2023: 86; 2024: 65; 2025: 57; 2026: 35.

265 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  6. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  7. 2023Cognitive Architectures for Language Agentscited by 422
  8. 2023Augmented Language Models: a Surveycited by 398
  9. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  10. 2023Large Language Models as Tool Makerscited by 293
  11. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  12. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
38 more
  1. 2024Token-Budget-Aware LLM Reasoningcited by 242
  2. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  3. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  4. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  5. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  6. 2022Can Foundation Models Wrangle Your Data?cited by 173
  7. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  8. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  9. 2022Evaluating and Inducing Personality in Pre-trained Language Modelscited by 122
  10. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  11. 2023Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakescited by 101
  12. 2023Cumulative Reasoning with Large Language Modelscited by 94
  13. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  14. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
  15. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  16. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  17. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  18. 2025Interactive Debugging and Steering of Multi-Agent AI Systemscited by 68
  19. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 67
  20. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  21. 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagramscited by 65
  22. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 61
  23. 2023AI-Generated Content (AIGC): A Surveycited by 59
  24. 2023PromptMagician: Interactive Prompt Engineering for Text-to-Image Creationcited by 58
  25. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  26. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  27. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 54
  28. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  29. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  30. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  31. 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
  32. 2023Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Datacited by 48
  33. 2024The Oscars of AI Theater: A Survey on Role-Playing with Language Modelscited by 47
  34. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  35. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 44
  36. 2022How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Modelscited by 43
  37. 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
  38. 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41

50 most cited of 265 All 265 (JSON)