# Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding

arXiv: https://arxiv.org/abs/2508.00420 · HTML: https://mlsanity.com/paper/2508.00420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 1 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [All-but-the-Top: Simple and Effective Postprocessing for Word Representations](https://mlsanity.com/paper/1702.01417/) (2017)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Evaluating Word Embedding Models: Methods and Experimental Results](https://mlsanity.com/paper/1901.09785/) (2019)
- [Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation](https://mlsanity.com/paper/1902.08850/) (2019)
- [Efficient Sentence Embedding using Discrete Cosine Transform](https://mlsanity.com/paper/1909.03104/) (2019)
- [Sentence Analogies: Exploring Linguistic Relationships and Regularities in Sentence Embeddings](https://mlsanity.com/paper/2003.04036/) (2020)

## Abstract (arXiv)

Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in Image and Signal processing suggest that wavelets can be effectively applied to Natural Language Processing (NLP) tasks that capture a variety of linguistic properties. In this paper, we leverage the power of applying Discrete Wavelet Transforms (DWT) to word and sentence embeddings. We first evaluate, intrinsically and extrinsically, how wavelets can effectively be used to consolidate important information in a word vector while reducing its dimensionality. We further combine DWT with Discrete Cosine Transform (DCT) to propose a non-parameterized model that compresses a sentence with a dense amount of information in a fixed size vector based on locally varying word features. We show the efficacy of the proposed paradigm on downstream applications models yielding comparable and even superior (in some tasks) results to original embeddings.