# Word2Spike: Poisson Rate Coding for Associative Memories and Neuromorphic Algorithms

arXiv: https://arxiv.org/abs/2509.07361 · HTML: https://mlsanity.com/paper/2509.07361/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks](https://mlsanity.com/paper/2302.13939/) (2023)
- [Rate Coding or Direct Coding: Which One is Better for Accurate, Robust, and Energy-efficient Spiking Neural Networks?](https://mlsanity.com/paper/2202.03133/) (2022)
- [Exploiting Noise as a Resource for Computation and Learning in Spiking Neural Networks](https://mlsanity.com/paper/2305.16044/) (2023)
- [SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation](https://mlsanity.com/paper/2308.15122/) (2023)
- [Episodic Memories Generation and Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2501.13121/) (2025)
- [Near-lossless Binarization of Word Embeddings](https://mlsanity.com/paper/1803.09065/) (2018)
- [Noise-Robust Deep Spiking Neural Networks with Temporal Information](https://mlsanity.com/paper/2104.11169/) (2021)
- [Reinforcement Learning with Low-Complexity Liquid State Machines](https://mlsanity.com/paper/1906.01695/) (2019)
- [Faster Binary Embeddings for Preserving Euclidean Distances](https://mlsanity.com/paper/2010.00712/) (2020)
- [Spiking representation learning for associative memories](https://mlsanity.com/paper/2406.03054/) (2024)
- [Stochastic Spiking Neural Networks with First-to-Spike Coding](https://mlsanity.com/paper/2404.17719/) (2024)

## Abstract (arXiv)

Spiking neural networks offer a promising path toward energy-efficient, brain-like associative memory. This paper introduces Word2Spike, a novel rate coding mechanism that combines continuous word embeddings and neuromorphic architectures. We develop a one-to-one mapping that converts multi-dimensional word vectors into spike-based attractor states using Poisson processes. Using BitNet b1.58 quantization, we maintain 97% semantic similarity of continuous embeddings on SimLex-999 while achieving 100% reconstruction accuracy on 10,000 words from OpenAI's text-embedding-3-large. We preserve analogy performance (100% of original embedding performance) even under intentionally introduced noise, indicating a resilient mechanism for semantic encoding in neuromorphic systems. Next steps include integrating the mapping with spiking transformers and liquid state machines (resembling Hopfield Networks) for further evaluation.