# LLMCRIT: Teaching Large Language Models to Use Criteria

arXiv: https://arxiv.org/abs/2403.01069 · HTML: https://mlsanity.com/paper/2403.01069/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 2 Mar 2024.

## Builds on (most cited first, 14 of 14)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision](https://mlsanity.com/paper/2305.03047/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs](https://mlsanity.com/paper/2303.16434/) (2023)
- [Shepherd: A Critic for Language Model Generation](https://mlsanity.com/paper/2308.04592/) (2023)
- [Specific versus General Principles for Constitutional AI](https://mlsanity.com/paper/2310.13798/) (2023)
- [Align on the Fly: Adapting Chatbot Behavior to Established Norms](https://mlsanity.com/paper/2312.15907/) (2023)

## Built on by (most cited first, 9 of 9)

- [TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation](https://mlsanity.com/paper/2410.03608/) (2024)
- [Improving Reward Models with Synthetic Critiques](https://mlsanity.com/paper/2405.20850/) (2024)
- [What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use](https://mlsanity.com/paper/2409.08775/) (2024)
- [Generative AI Act II: Test Time Scaling Drives Cognition Engineering](https://mlsanity.com/paper/2504.13828/) (2025)
- [Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback](https://mlsanity.com/paper/2507.16007/) (2025)
- [LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information](https://mlsanity.com/paper/2502.02095/) (2025)
- [SAFETY-J: Evaluating Safety with Critique](https://mlsanity.com/paper/2407.17075/) (2024)
- [LLMs Behind the Scenes: Enabling Narrative Scene Illustration](https://mlsanity.com/paper/2509.22940/) (2025)
- [Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments?](https://mlsanity.com/paper/2605.29857/) (2026)

## Abstract (arXiv)

Humans follow criteria when they execute tasks, and these criteria are directly used to assess the quality of task completion. Therefore, having models learn to use criteria to provide feedback can help humans or models to perform tasks better. However, existing research in this field tends to consider only a limited set of criteria or quality assessment aspects. To fill this gap, we propose a general framework that enables large language models (LLMs) to use comprehensive criteria for a task in delivering natural language feedback on task execution. In particular, we present a model-in-the-loop framework that semi-automatically derives criteria from collected guidelines for different writing tasks and constructs in-context demonstrations for each criterion. We choose three tasks from real-world scenarios to operationalize this idea: paper introduction writing, Python code writing, and Reddit post writing, and evaluate our feedback generation framework using different LLMs. The results reveal the fine-grained effects of incorporating criteria and demonstrations and provide valuable insights on how to teach LLMs to use criteria more effectively.