mlsanityData to 14 Oct 2026

Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition

arXiv 2606.13379 ↗First posted 11 Jun 2026cs.LG cs.AR cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 33 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

15 of 33 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  6. 2018Self-Attention with Relative Position Representationscited by 1,120
  7. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  9. 2018RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbarscited by 19
  10. 2016RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networkscited by 16
  11. 2024Synaptogen: A cross-domain generative device model for large-scale neuromorphic circuit designcited by 2
  12. 2024Neuromorphic Circuit Simulation with Memristors: Design and Evaluation Using MemTorch for MNIST and CIFARcited by 2
3 more
  1. 2025Supplementary Resources and Analysis for Automatic Speech Recognition Systems Trained on the Loquacious Datasetcited by 2
  2. 2019A Transformer with Interleaved Self-attention and Convolution for Hybrid Acoustic Modelscited by 1
  3. 2025Running Conventional Automatic Speech Recognition on Memristor Hardware: A Simulated Approachcited by 1

Built on by

Not cited yet.