# Lightweight Convolutional Representations for On-Device Natural Language Processing

arXiv: https://arxiv.org/abs/2002.01535 · HTML: https://mlsanity.com/paper/2002.01535/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2020.

## Built on by (most cited first, 4 of 4)

- [Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog](https://mlsanity.com/paper/2104.04923/) (2021)
- [Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research](https://mlsanity.com/paper/2104.07874/) (2021)
- [LIDSNet: A Lightweight on-device Intent Detection model using Deep Siamese Network](https://mlsanity.com/paper/2110.15717/) (2021)
- [TEESlice: Protecting Sensitive Neural Network Models in Trusted Execution Environments When Attackers have Pre-Trained Models](https://mlsanity.com/paper/2411.09945/) (2024)

## Abstract (arXiv)

The increasing computational and memory complexities of deep neural networks have made it difficult to deploy them on low-resource electronic devices (e.g., mobile phones, tablets, wearables). Practitioners have developed numerous model compression methods to address these concerns, but few have condensed input representations themselves. In this work, we propose a fast, accurate, and lightweight convolutional representation that can be swapped into any neural model and compressed significantly (up to 32x) with a negligible reduction in performance. In addition, we show gains over recurrent representations when considering resource-centric metrics (e.g., model file size, latency, memory usage) on a Samsung Galaxy S9.