mlsanityData to 7 Oct 2026

ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles

arXiv 2310.15428 ↗First posted 24 Oct 2023cs.HC cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) prompting is a promising new approach for users to create and customize their own chatbots. However, current methods for steering a chatbot's outputs, such as prompt engineering and fine-tuning, do not support users in converting their natural feedback on the model's outputs to changes in the prompt or model. In this work, we explore how to enable users to interactively refine model outputs through their feedback, by helping them convert their feedback into a set of principles (i.e. a constitution) that dictate the model's behavior. From a formative study, we (1) found that users needed support converting their feedback into principles for the chatbot and (2) classified the different principle types desired by users. Inspired by these findings, we developed ConstitutionMaker, an interactive tool for converting user feedback into principles, to steer LLM-based chatbots. With ConstitutionMaker, users can provide either positive or negative feedback in natural language, select auto-generated feedback, or rewrite the chatbot's response; each mode of feedback automatically generates a principle that is inserted into the chatbot's prompt. In a user study with 14 participants, we compare ConstitutionMaker to an ablated version, where users write their own principles. With ConstitutionMaker, participants felt that their principles could better guide the chatbot, that they could more easily convert their feedback into principles, and that they could write principles more efficiently, with less mental demand. ConstitutionMaker helped users identify ways to improve the chatbot, formulate their intuitive responses to the model into feedback, and convert this feedback into specific and clear principles. Together, these findings inform future tools that support the interactive critiquing of LLM outputs.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 22 of its 63 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

22 of 63 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  5. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  6. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  7. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  8. 2017Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memorycited by 167
  9. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  10. 2016Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbotscited by 103
  11. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  12. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
10 more
  1. 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
  2. 2023"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Modelscited by 48
  3. 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
  4. 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41
  5. 2022Opal: Multimodal Image Generation for News Illustrationcited by 36
  6. 2021On Training Instance Selection for Few-Shot Neural Text Generationcited by 25
  7. 2021PopBlends: Strategies for Conceptual Blending with Large Language Modelscited by 22
  8. 2023ScatterShot: Interactive In-context Example Curation for Text Transformationcited by 20
  9. 2023PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Modelscited by 19
  10. 2019Multi-modal Synthesis of Regular Expressionscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 10; 2026: 5.

33 papers.

Sort:

By area:

  1. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
  2. 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 72
  3. 2025Interactive Debugging and Steering of Multi-Agent AI Systemscited by 68
  4. 2024Inverse Constitutional AI: Compressing Preferences into Principlescited by 39
  5. 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
  6. 2024What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Usecited by 24
  7. 2024Designing for Human-Agent Alignment: Understanding what humans want from their agentscited by 18
  8. 2024HonestLLM: Toward an Honest and Helpful Large Language Modelcited by 17
  9. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  10. 2024Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflowcited by 13
  11. 2024Social Skill Training with Large Language Modelscited by 13
  12. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AIcited by 13
21 more
  1. 2024Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Productscited by 11
  2. 2024ConstitutionalExperts: Training a Mixture of Principle-based Promptscited by 7
  3. 2024Self-Directed Synthetic Dialogues and Revisions Technical Reportcited by 7
  4. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  5. 2024Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviorscited by 5
  6. 2025SPRI: Aligning Large Language Models with Context-Situated Principlescited by 5
  7. 2024Survey of User Interface Design and Interaction Techniques in Generative AI Applicationscited by 3
  8. 2025Human-AI Complementarity: A Goal for Amplified Oversightcited by 3
  9. 2025FeedQUAC: Quick Unobtrusive AI-Generated Commentarycited by 2
  10. 2026Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Editscited by 2
  11. 2023Contextual Confidence and Generative AIcited by 1
  12. 2024POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Modelscited by 1
  13. 2025Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoningcited by 1
  14. 2025DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflowcited by 1
  15. 2025Botender: Supporting Communities in Collaboratively Designing AI Agents through Case-Based Provocationscited by 1
  16. 2025Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM Behaviorcited by 1
  17. 2026LLM Prompt Evaluation for Educational Applicationscited by 1
  18. 2026Generative Experiences for Digital Mental Health Interventions: Evidence from a Randomized Studycited by 1
  19. 2025PolicyPad: Collaborative Prototyping of LLM Policiescited by 0
  20. 2026Bridging the Intention-Expression Gap: Aligning Multi-Dimensional Preferences via Hierarchical Relevance Feedback in Text-to-Image Diffusioncited by 0
  21. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0