mlsanityData to 14 Oct 2026

Embedding Compression for Text Classification Using Dictionary Screening

arXiv 2211.12715 ↗First posted 23 Nov 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a dictionary screening method for embedding compression in text classification tasks. The key purpose of this method is to evaluate the importance of each keyword in the dictionary. To this end, we first train a pre-specified recurrent neural network-based model using a full dictionary. This leads to a benchmark model, which we then use to obtain the predicted class probabilities for each sample in a dataset. Next, to evaluate the impact of each keyword in affecting the predicted class probabilities, we develop a novel method for assessing the importance of each keyword in a dictionary. Consequently, each keyword can be screened, and only the most important keywords are reserved. With these screened keywords, a new dictionary with a considerably reduced size can be constructed. Accordingly, the original text sequence can be substantially compressed. The proposed method leads to significant reductions in terms of parameters, average text sequence, and dictionary size. Meanwhile, the prediction power remains very competitive compared to the benchmark model. Extensive numerical studies are presented to demonstrate the empirical performance of the proposed method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

34 of 59 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  10. 2016FastText.zip: Compressing text classification modelscited by 335
  11. 2013Predicting Parameters in Deep Learningcited by 224
  12. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
22 more
  1. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  2. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  3. 2018SGM: Sequence Generation Model for Multi-label Classificationcited by 89
  4. 2013Revisiting the Nystrom Method for Improved Large-Scale Machine Learningcited by 77
  5. 2017Simple and Effective Dimensionality Reduction for Word Embeddingscited by 39
  6. 2017An Entropy-based Pruning Method for CNN Compressioncited by 37
  7. 2017Compressing Word Embeddings via Deep Compositional Code Learningcited by 35
  8. 2018The unreasonable effectiveness of the forget gatecited by 34
  9. 2018Alternating Multi-bit Quantization for Recurrent Neural Networkscited by 31
  10. 2017Compressing Recurrent Neural Network with Tensor Traincited by 28
  11. 2020Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Functioncited by 23
  12. 2018Online Embedding Compression for Text Classification using Low Rank Matrix Factorizationcited by 22
  13. 2018GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Trainingcited by 21
  14. 2018Grow and Prune Compact, Fast, and Accurate LSTMscited by 20
  15. 2015Self-Adaptive Hierarchical Sentence Modelcited by 10
  16. 2016On the Compression of Recurrent Neural Networks with an Application to LVCSR acoustic modeling for Embedded Speech Recognitioncited by 10
  17. 2018A novel channel pruning method for deep neural network compressioncited by 8
  18. 2019Compression of Recurrent Neural Networks for Efficient Language Modelingcited by 7
  19. 2016Efficient Character-level Document Classification by Combining Convolution and Recurrent Layerscited by 6
  20. 2018Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Trainingcited by 5
  21. 2020SentiLSTM: A Deep Learning Approach for Sentiment Analysis of Restaurant Reviewscited by 4
  22. 2016A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Featurescited by 3

Built on by

Not cited yet.