mlsanityData to 14 Oct 2026

Low-Precision Streaming PCA

arXiv 2510.22440 ↗First posted 25 Oct 2025cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Low-precision streaming PCA estimates the top principal component in a streaming setting under limited precision. We establish an information-theoretic lower bound on the quantization resolution required to achieve a target accuracy for the leading eigenvector. We study Oja's algorithm for streaming PCA under linear and nonlinear stochastic quantization. The quantized variants use unbiased stochastic quantization of the weight vector and the updates. Under mild moment and spectral-gap assumptions on the data distribution, we show that a batched version achieves the lower bound up to logarithmic factors under both schemes. This leads to a nearly dimension-free quantization error in the nonlinear quantization setting. Empirical evaluations on synthetic streams validate our theoretical findings and demonstrate that our low-precision methods closely track the performance of standard Oja's algorithm.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 68 references that are in the map. First posted on arXiv 25 Oct 2025.

Builds on

37 of 68 references.

References · 37

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  2. 2017Mixed Precision Trainingcited by 1,156
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  4. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  5. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  6. 2017Training Quantized Nets: A Deeper Understandingcited by 75
  7. 2019SGD without Replacement: Sharper Rates for General Smooth Convex Functionscited by 61
  8. 2013The Noisy Power Method: A Meta Algorithm with Applicationscited by 58
  9. 2018Mixed Precision Training of Convolutional Neural Networks using Integer Operationscited by 45
  10. 2017Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networkscited by 44
  11. 2016First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Ratecited by 32
  12. 2015Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithmscited by 31
25 more
  1. 2021FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Roundingcited by 31
  2. 2016Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithmcited by 29
  3. 2022Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networkscited by 26
  4. 2014Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problemscited by 25
  5. 2016An Improved Gap-Dependency Analysis of the Noisy Power Methodcited by 24
  6. 2017Accelerated Stochastic Power Iterationcited by 22
  7. 2018High-Accuracy Low-Precision Trainingcited by 20
  8. 2022Semi-Random Sparse Recovery in Nearly-Linear Timecited by 18
  9. 2015Convergence of Stochastic Gradient Descent for PCAcited by 17
  10. 2015The Fast Convergence of Incremental PCAcited by 15
  11. 2019AdaOja: Adaptive Learning Rates for Streaming PCAcited by 11
  12. 2021On the Optimality of the Oja's Algorithm for Online PCAcited by 11
  13. 2021Bootstrapping the error of Oja's algorithmcited by 11
  14. 2024Collage: Light-Weight Low-Precision Strategy for LLM Trainingcited by 10
  15. 2025Stochastic Rounding for LLM Training: Theory and Practicecited by 10
  16. 2015Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexitycited by 9
  17. 2018History PCA: A New Algorithm for Streaming PCAcited by 9
  18. 2023Streaming PCA for Markovian Datacited by 9
  19. 2024Oja's Algorithm for Streaming Sparse PCAcited by 9
  20. 2024Black-Box k-to-1-PCA Reductions: Theory and Applicationscited by 7
  21. 2018Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimizationcited by 5
  22. 2022On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computationcited by 4
  23. 2025Beyond Sin-Squared Error: Linear-Time Entrywise Uncertainty Quantification for Streaming PCAcited by 3
  24. 2023DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inferencecited by 2
  25. 2024BitQ: Tailoring Block Floating Point Precision for Improved DNN Efficiency on Resource-Constrained Devicescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Gap-Free Streaming PCA Beyond Rank-One Updates: Near-Optimal Rates and Applications to Differential Privacycited by 1
  2. 2026PCA-Driven Adaptive Sensor Triage for Edge AI Inferencecited by 0
  3. 2026Rotation-Based Subspace Tracking for Robust Kernel PCA on Streaming Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Distributed Principal Component Analysis with Limited Communication
  2. 2026Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
  3. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
  4. 2026Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization
  5. 2026Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape
  6. 2026Price of metric universality in vector quantization is at most 0.11 bit
  7. 2026Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization
  8. 2026PCA-VAE: Differentiable Subspace Quantization without Codebook Collapse