# AnySimLite: A Lightweight Few-Shot Similarity Encoder for On-Device Speech-Adjacent Classification

arXiv: https://arxiv.org/abs/2606.26452 · HTML: https://mlsanity.com/paper/2606.26452/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Natural Language Understanding with the Quora Question Pairs Dataset](https://mlsanity.com/paper/1907.01041/) (2019)
- [Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale](https://mlsanity.com/paper/2411.05045/) (2024)
- [A Machine Learning Approach to Comment Toxicity Classification](https://mlsanity.com/paper/1903.06765/) (2019)
- [RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis](https://mlsanity.com/paper/2406.00367/) (2024)
- [LIDSNet: A Lightweight on-device Intent Detection model using Deep Siamese Network](https://mlsanity.com/paper/2110.15717/) (2021)
- [ESIE-BERT: Enriching Sub-words Information Explicitly with BERT for Joint Intent Classification and SlotFilling](https://mlsanity.com/paper/2211.14829/) (2022)

## Abstract (arXiv)

To minimize privacy concerns and inference latency on edge devices like smartphones, lightweight on-device models remain important for end-user applications. Many of these applications involve natural language classification, but deploying multiple specialized models creates a memory footprint challenge. We investigate: Can a single lightweight architecture solve multiple Speech-Adjacent (SA) classification tasks through reduction to a nuanced text similarity formulation? We propose AnySimLite, a lightweight similarity encoder that combines word-level and character-level channels. Together with a dataset transformation strategy, we evaluate AnySimLite across multiple SA classification tasks and show that it consistently achieves state-of-the-art (SOTA) or SOTA-competitive performance in few-shot settings while maintaining a low memory footprint. Even in the worst case, the performance drop remains below 7% while using <frac1250th of the model size of the SOTA qLLaMALoRA-7B baseline.