# Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2606.13379 · HTML: https://mlsanity.com/paper/2606.13379/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars](https://mlsanity.com/paper/1809.00072/) (2018)
- [RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks](https://mlsanity.com/paper/1608.00895/) (2016)
- [Synaptogen: A cross-domain generative device model for large-scale neuromorphic circuit design](https://mlsanity.com/paper/2404.06344/) (2024)
- [Neuromorphic Circuit Simulation with Memristors: Design and Evaluation Using MemTorch for MNIST and CIFAR](https://mlsanity.com/paper/2407.13410/) (2024)
- [Supplementary Resources and Analysis for Automatic Speech Recognition Systems Trained on the Loquacious Dataset](https://mlsanity.com/paper/2512.17915/) (2025)
- [A Transformer with Interleaved Self-attention and Convolution for Hybrid Acoustic Models](https://mlsanity.com/paper/1910.10352/) (2019)
- [Running Conventional Automatic Speech Recognition on Memristor Hardware: A Simulated Approach](https://mlsanity.com/paper/2505.24721/) (2025)

## Abstract (arXiv)

Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.