mlsanityData to 14 Oct 2026

Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction

arXiv 2504.03159 ↗First posted 4 Apr 2025cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prompt can cause significant discrepancies in model performance. We attribute this prompt brittleness largely to the narrow focus on nexttoken probabilities in existing methods. To address this, we propose Placeholding Parallel Prediction (P3), a novel approach that predicts token probabilities across multiple positions and simulates comprehensive sampling of generation paths in a single run of a language model. Experiments show improved accuracy and up to 98% reduction in the standard deviation across prompts, boosting robustness. Even without a prompt, P3 maintains comparable performance, reducing the need for prompt engineering.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

31 of 52 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
19 more
  1. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  3. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  5. 2018Universal Sentence Encodercited by 489
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  7. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  8. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 263
  9. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  10. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  11. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  12. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  13. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  14. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 94
  15. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  16. 2024UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulationcited by 31
  17. 2023TENT: Connect Language Models with IoT Sensors for Zero-Shot Activity Recognitioncited by 25
  18. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  19. 2024Nash CoT: Multi-Path Inference with Preference Equilibriumcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025GVPO: Group Variance Policy Optimization for Large Language Model Post-Trainingcited by 32
  2. 2025Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notationcited by 3
  3. 2026Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecificationcited by 3
  4. 2026Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Predictioncited by 3
  5. 2025The Gaining Paths to Investment Success: Information-Driven LLM Graph Reasoning for Venture Capital Predictioncited by 1
  6. 2026Enforcing LLM Safety through DMD-based Classification of Prompt-Response Embedding Dynamicscited by 0
  7. 2026GVPO++: Group Variance Policy Optimization for LLM Post-Training and On-Policy Distillationcited by 0