mlsanityData to 7 Oct 2026

Interactive Prompt Debugging with Sequence Salience

arXiv 2404.07498 ↗First posted 11 Apr 2024cs.CL cs.AI cs.HCcited by 6

Not on the map. Why?

Abstract (arXiv)

We present Sequence Salience, a visual tool for interactive prompt debugging with input salience methods. Sequence Salience builds on widely used salience methods for text classification and single-token prediction, and extends this to a system tailored for debugging complex LLM prompts. Our system is well-suited for long texts, and expands on previous work by 1) providing controllable aggregation of token-level salience to the word, sentence, or paragraph level, making salience over long inputs tractable; and 2) supporting rapid iteration where practitioners can act on salience results, refine prompts, and run salience on the new output. We include case studies showing how Sequence Salience can help practitioners work with several complex prompting strategies, including few-shot, chain-of-thought, and constitutional principles. Sequence Salience is built on the Learning Interpretability Tool, an open-source platform for ML model visualizations, and code, notebooks, and tutorials are available at http://goo.gle/sequence-salience.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

30 of 43 references.

References · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  10. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
18 more
  1. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  2. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  4. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  5. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  6. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  7. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  8. 2018Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Modelscited by 69
  9. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  10. 2019Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AIcited by 63
  11. 2018What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Playcited by 52
  12. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  13. 2022What do tokens know about their characters and how do they know it?cited by 36
  14. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  15. 2023Using Captum to Explain Generative Language Modelscited by 33
  16. 2021LMdiff: A Visual Diff Tool to Compare Language Modelscited by 17
  17. 2024LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanationscited by 16
  18. 2020The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsetscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  2. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
  3. 2026DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviorscited by 7
  4. 2025Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Modelscited by 2
  5. 2026Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analyticscited by 1
  6. 2025PromptFlow: Training Prompts Like Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Prompt-Driven Code Summarization: A Systematic Literature Review
  2. 2025The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance
  3. 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions
  4. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
  5. 2024POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
  6. 2025Empirical Prompt Engineering for Construct Identification with Large Language Models
  7. 2026Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs
  8. 2025Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach