Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training
Not on the map. Why?
Abstract (arXiv)
Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. However, non-ideal analog device properties make the training on AIMC devices challenging. In particular, its update asymmetry can induce a systematic drift of weight updates towards a device-specific symmetric point (SP), which typically does not align with the optimum of the training objective. To mitigate this bias, most existing works assume the SP is known and pre-calibrate it to zero before training by setting the reference point as the SP. Nevertheless, calibrating AIMC devices requires costly pulse updates, and residual calibration error can directly degrade training performance. In this work, we present the first theoretical characterization of the pulse complexity of SP calibration and the resulting estimation error. We further propose a dynamic SP estimation method that tracks the SP during model training, and establishes its convergence guarantees. In addition, we develop an enhanced variant based on chopping and filtering techniques from digital signal processing. Numerical experiments demonstrate both the efficiency and effectiveness of the proposed method.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 63 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
23 of 63 references.
References · 23
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
- 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
- 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
- 2018Approximation Methods for Bilevel Programmingcited by 236
- 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 234
- 2016Hyperparameter optimization with approximate gradientcited by 233
- 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 222
11 more
- 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202
- 2021A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrayscited by 39
- 2016Acceleration of Deep Neural Network Training with Resistive Cross-Point Devicescited by 29
- 2018Multi-Level Stochastic Gradient Methods for Nested Composition Optimizationcited by 24
- 2019Algorithm for Training Neural Networks on Resistive Device Arrayscited by 19
- 2022A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequencescited by 17
- 2023Fast offset corrected in-memory trainingcited by 11
- 2019Zero-shifting Technique for Deep Neural Network Training on Resistive Cross-point Arrayscited by 7
- 2024Towards Exact Gradient-based Training on Analog In-memory Computingcited by 7
- 2025Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functionscited by 5
- 2025In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learningcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.