# What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns

arXiv: https://arxiv.org/abs/2504.15815 · HTML: https://mlsanity.com/paper/2504.15815/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 83 references that are in the map.
- First posted on arXiv 22 Apr 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Do Prompt-Based Models Really Understand the Meaning of their Prompts?](https://mlsanity.com/paper/2109.01247/) (2021)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild](https://mlsanity.com/paper/2406.04770/) (2024)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [RAIN: Your Language Models Can Align Themselves without Finetuning](https://mlsanity.com/paper/2309.07124/) (2023)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments](https://mlsanity.com/paper/2311.09718/) (2023)
- [Robustness of Learning from Task Instructions](https://mlsanity.com/paper/2212.03813/) (2022)
- [Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning](https://mlsanity.com/paper/2302.04732/) (2023)
- [Supporting Sensemaking of Large Language Model Outputs at Scale](https://mlsanity.com/paper/2401.13726/) (2024)
- [Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development](https://mlsanity.com/paper/2110.02932/) (2021)
- [What Makes Math Word Problems Challenging for LLMs?](https://mlsanity.com/paper/2403.11369/) (2024)
- [Label-Descriptive Patterns and Their Application to Characterizing Classification Errors](https://mlsanity.com/paper/2110.09599/) (2021)
- [What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research](https://mlsanity.com/paper/2012.07619/) (2020)

## Built on by (most cited first, 4 of 4)

- [MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs](https://mlsanity.com/paper/2505.21693/) (2025)
- [Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions](https://mlsanity.com/paper/2512.12775/) (2025)
- [EMBRACE: Shaping Inclusive Opinion Representation by Aligning Implicit Conversations with Social Norms](https://mlsanity.com/paper/2507.20264/) (2025)
- [InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation](https://mlsanity.com/paper/2512.10234/) (2025)

## Abstract (arXiv)

Prompt engineering for large language models is challenging, as even small prompt perturbations or model changes can significantly impact the generated output texts. Existing evaluation methods of LLM outputs, either automated metrics or human evaluation, have limitations, such as providing limited insights or being labor-intensive. We propose Spotlight, a new approach that combines both automation and human analysis. Based on data mining techniques, we automatically distinguish between random (decoding) variations and systematic differences in language model outputs. This process provides token patterns that describe the systematic differences and guide the user in manually analyzing the effects of their prompts and changes in models efficiently. We create three benchmarks to quantitatively test the reliability of token pattern extraction methods and demonstrate that our approach provides new insights into established prompt data. From a human-centric perspective, through demonstration studies and a user study, we show that our token pattern approach helps users understand the systematic differences of language model outputs. We are further able to discover relevant differences caused by prompt and model changes (e.g. related to gender or culture), thus supporting the prompt engineering process and human-centric model behavior research.