mlsanityData to 7 Oct 2026

SALSA: Single-pass Autoregressive LLM Structured Classification

arXiv 2510.22691 ↗First posted 26 Oct 2025cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Despite their impressive generalization capabilities, instruction-tuned Large Language Models often underperform on text classification benchmarks. We introduce SALSA, a coherent pipeline that combines structured prompting, class-to-token mapping, and parameter-efficient fine-tuning, thereby avoiding cold-start training. Each class label is mapped to a distinct output token, and prompts are constructed to elicit a single-token response. During inference, the model's output is projected only onto the logits of the relevant class tokens, enabling efficient and accurate classification in a single forward pass. SALSA achieves state-of-the-art results across diverse benchmarks, demonstrating its robustness and scalability for LLM-based classification applications.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 39 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

28 of 39 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
16 more
  1. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  2. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  8. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  9. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  11. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  12. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  13. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  14. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  15. 2024Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classificationcited by 66
  16. 2023Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUEcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detectorcited by 2