# GLiClass: Generalist Lightweight Model for Sequence Classification Tasks arXiv: https://arxiv.org/abs/2508.07662 ยท HTML: https://mlsanity.com/paper/2508.07662/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 30 references that are in the map. - First posted on arXiv 11 Aug 2025. ## Builds on (most cited first, 20 of 20) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019) - [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013) - [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022) - [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020) - [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019) - [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021) - [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024) - [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019) - [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022) - [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://mlsanity.com/paper/2311.08526/) (2023) - [A Survey on Text Classification: From Shallow to Deep Learning](https://mlsanity.com/paper/2008.00364/) (2020) - [Zero-shot Text Classification With Generative Language Models](https://mlsanity.com/paper/1912.10165/) (2019) - [Seq vs Seq: An Open Suite of Paired Encoders and Decoders](https://mlsanity.com/paper/2507.11412/) (2025) - [Evaluation of sentence embeddings in downstream and linguistic probing tasks](https://mlsanity.com/paper/1806.06259/) (2018) - [Building Efficient Universal Classifiers with Natural Language Inference](https://mlsanity.com/paper/2312.17543/) (2023) - [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022) - [GLiREL -- Generalist Model for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2501.03172/) (2025) - [GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks](https://mlsanity.com/paper/2406.12925/) (2024) - [GLiDRE: Generalist Lightweight model for Document-level Relation Extraction](https://mlsanity.com/paper/2508.00757/) (2025) ## Built on by (most cited first, 15 of 15) - [GLiGuard: Schema-Conditioned Classification for LLM Safeguard](https://mlsanity.com/paper/2605.07982/) (2026) - [Typed Decision Models: An Early Evidence Audit and Evaluation Checklist](https://mlsanity.com/paper/2609.32160/) (2026) - [GLiNER Guard: Unified Encoder Family for Production LLM Safety and Privacy](https://mlsanity.com/paper/2605.05277/) (2026) - [LAVOIR: Teaching a Single-Pass Decision Encoder When and What to Ask with Amortized Value of Information](https://mlsanity.com/paper/2609.30706/) (2026) - [Benchmarking System One decision models against trained classifiers and language models for automated decision gates](https://mlsanity.com/paper/2610.00346/) (2026) - [SearchJev: A Fast and Calibrated System-1 Model for Search Agents](https://mlsanity.com/paper/2610.05107/) (2026) - [Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification](https://mlsanity.com/paper/2601.16530/) (2026) - [Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content](https://mlsanity.com/paper/2605.29659/) (2026) - [SAGE: A Unified Algebra and Self-Adaptive Execution for AI Functions in SQL](https://mlsanity.com/paper/2608.20630/) (2026) - [SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology](https://mlsanity.com/paper/2609.02292/) (2026) - [Chinese-Jev: Bringing System One Model to Chinese-Language Tasks](https://mlsanity.com/paper/2609.36965/) (2026) - [When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows](https://mlsanity.com/paper/2610.13092/) (2026) - [Releasing the Inner Jev: A Simple Recipe for Open Decision Models](https://mlsanity.com/paper/2610.17205/) (2026) - [AgriJev: Efficient Agricultural Candidate Scoring with Multi-Scale Adapted Models](https://mlsanity.com/paper/2610.19478/) (2026) - [Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters](https://mlsanity.com/paper/2610.19901/) (2026) ## Abstract (arXiv) Classification is one of the most widespread tasks in AI applications, serving often as the first step in filtering, sorting, and categorizing data. Since modern AI systems must handle large volumes of input data and early pipeline stages can propagate errors downstream, achieving high efficiency and accuracy is critical. Moreover, classification requirements can change dynamically based on user needs, necessitating models with strong zero-shot capabilities. While generative LLMs have become mainstream for zero-shot classification due to their versatility, they suffer from inconsistent instruction following and computational inefficiency. Cross-encoders, commonly used as rerankers in RAG pipelines, face a different bottleneck: they must process text-label pairs sequentially, significantly reducing efficiency with large label sets. Embedding-based approaches offer good efficiency but struggle with complex scenarios involving logical and semantic constraints. We propose GLiClass, a novel method that adapts the GLiNER architecture for sequence classification tasks. Our approach achieves strong accuracy and efficiency comparable to embedding-based methods, while maintaining the flexibility needed for zero-shot and few-shot learning scenarios. Additionally, we adapted proximal policy optimization (PPO) for multi-label text classification, enabling training classifiers in data-sparse conditions or from human feedback.