# Can Large Language Models Follow Concept Annotation Guidelines? A Case Study on Scientific and Financial Domains

arXiv: https://arxiv.org/abs/2311.08704 · HTML: https://mlsanity.com/paper/2311.08704/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://mlsanity.com/paper/2305.12295/) (2023)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts](https://mlsanity.com/paper/1710.06071/) (2017)
- [Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge](https://mlsanity.com/paper/2305.01651/) (2023)

## Built on by (most cited first, 7 of 7)

- [VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data](https://mlsanity.com/paper/2502.06737/) (2025)
- [Fearful Falcons and Angry Llamas: Emotion Category Annotations of Arguments by Humans and LLMs](https://mlsanity.com/paper/2412.15993/) (2024)
- [Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration](https://mlsanity.com/paper/2501.12332/) (2025)
- [Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysis](https://mlsanity.com/paper/2503.16575/) (2025)
- [iPOE: Interpretable Prompt Optimization via Explanations](https://mlsanity.com/paper/2605.18113/) (2026)
- [Refining and Reusing Annotation Guidelines for LLM Annotation](https://mlsanity.com/paper/2605.20809/) (2026)
- [WhoSaidIt: Human-LLM Collaborative Annotation for Text-Based Multilingual Speaker-Attribute Classification](https://mlsanity.com/paper/2605.26070/) (2026)

## Abstract (arXiv)

Although large language models (LLMs) exhibit remarkable capacity to leverage in-context demonstrations, it is still unclear to what extent they can learn new concepts or facts from ground-truth labels. To address this question, we examine the capacity of instruction-tuned LLMs to follow in-context concept guidelines for sentence labeling tasks. We design guidelines that present different types of factual and counterfactual concept definitions, which are used as prompts for zero-shot sentence classification tasks. Our results show that although concept definitions consistently help in task performance, only the larger models (with 70B parameters or more) have limited ability to work under counterfactual contexts. Importantly, only proprietary models such as GPT-3.5 and GPT-4 can recognize nonsensical guidelines, which we hypothesize is due to more sophisticated alignment methods. Finally, we find that Falcon-180B-chat is outperformed by Llama-2-70B-chat is most cases, which indicates that careful fine-tuning is more effective than increasing model scale. Altogether, our simple evaluation method reveals significant gaps in concept understanding between the most capable open-source language models and the leading proprietary APIs.