# Evolutionary Feature-wise Thresholding for Binary Representation of NLP Embeddings

arXiv: https://arxiv.org/abs/2507.17025 · HTML: https://mlsanity.com/paper/2507.17025/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [BitNet: Scaling 1-bit Transformers for Large Language Models](https://mlsanity.com/paper/2310.11453/) (2023)
- [Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Models](https://mlsanity.com/paper/2406.12311/) (2024)
- [Near-lossless Binarization of Word Embeddings](https://mlsanity.com/paper/1803.09065/) (2018)
- [MinMax Radon Barcodes for Medical Image Retrieval](https://mlsanity.com/paper/1610.00318/) (2016)
- [Barcode Annotations for Medical Image Retrieval: A Preliminary Investigation](https://mlsanity.com/paper/1505.05212/) (2015)
- [Barcodes for Medical Image Retrieval Using Autoencoded Radon Transform](https://mlsanity.com/paper/1609.05112/) (2016)
- [Training Artificial Neural Networks by Coordinate Search Algorithm](https://mlsanity.com/paper/2402.12646/) (2024)
- [Radon Features and Barcodes for Medical Image Retrieval via SVM](https://mlsanity.com/paper/1604.04675/) (2016)
- [Newton-Type Optimal Thresholding Algorithms for Sparse Optimization Problems](https://mlsanity.com/paper/2104.02371/) (2021)

## Abstract (arXiv)

Efficient text embedding is crucial for large-scale natural language processing (NLP) applications, where storage and computational efficiency are key concerns. In this paper, we explore how using binary representations (barcodes) instead of real-valued features can be used for NLP embeddings derived from machine learning models such as BERT. Thresholding is a common method for converting continuous embeddings into binary representations, often using a fixed threshold across all features. We propose a Coordinate Search-based optimization framework that instead identifies the optimal threshold for each feature, demonstrating that feature-specific thresholds lead to improved performance in binary encoding. This ensures that the binary representations are both accurate and efficient, enhancing performance across various features. Our optimal barcode representations have shown promising results in various NLP applications, demonstrating their potential to transform text representation. We conducted extensive experiments and statistical tests on different NLP tasks and datasets to evaluate our approach and compare it to other thresholding methods. Binary embeddings generated using using optimal thresholds found by our method outperform traditional binarization methods in accuracy. This technique for generating binary representations is versatile and can be applied to any features, not just limited to NLP embeddings, making it useful for a wide range of domains in machine learning applications.