mlsanityData to 14 Oct 2026

Optimal whitening and decorrelation

arXiv 1512.00809 ↗First posted 2 Dec 2015stat.ME stat.MLcited by 97

Not on the map. Why?

Abstract (arXiv)

Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example based on principal component analysis (PCA), Cholesky matrix decomposition and zero-phase component analysis (ZCA), among others. Here we provide an overview of the underlying theory and discuss five natural whitening procedures. Subsequently, we demonstrate that investigating the cross-covariance and the cross-correlation matrix between sphered and original variables allows to break the rotational invariance and to identify optimal whitening transformations. As a result we recommend two particular approaches: ZCA-cor whitening to produce sphered variables that are maximally similar to the original variables, and PCA-cor whitening to obtain sphered variables that maximally compress the original variables.

Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 8; 2023: 19; 2024: 14; 2025: 12; 2026: 33.

97 papers.

Sort:

By area:

  1. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  2. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  3. 2021Generalized Shape Metrics on Neural Representationscited by 112
  4. 2023CamP: Camera Preconditioning for Neural Radiance Fieldscited by 55
  5. 2023CAP-VSTNet: Content Affinity Preserved Versatile Style Transfercited by 24
  6. 2021A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecastingcited by 21
  7. 2022An Investigation into Whitening Loss for Self-supervised Learningcited by 21
  8. 2024PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillationcited by 21
  9. 2021Robust Differentiable SVDcited by 19
  10. 2023Discovering Universal Geometry in Embeddings with ICAcited by 19
  11. 2023Sequential Memory with Temporal Predictive Codingcited by 16
  12. 2023Variance-Covariance Regularization Improves Representation Learningcited by 14
38 more
  1. 2021Stochastic Whitening Batch Normalizationcited by 9
  2. 2023A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Modelscited by 9
  3. 2023On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learningcited by 8
  4. 2024DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillationcited by 8
  5. 2026Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectorycited by 8
  6. 2022IsoVec: Controlling the Relative Isomorphism of Word Embedding Spacescited by 7
  7. 2023Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrievalcited by 7
  8. 2024Efficient Deep Learning with Decorrelated Backpropagationcited by 7
  9. 2026PPISP: Physically-Plausible Compensation and Control of Photometric Variations in Radiance Field Reconstructioncited by 7
  10. 2026Gauge-invariant representation holonomycited by 7
  11. 2021Interpretability in deep learning for finance: a case study for the Heston modelcited by 6
  12. 2021Inverting Adversarially Robust Networks for Image Synthesiscited by 6
  13. 2023D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theorycited by 6
  14. 2022Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptationcited by 4
  15. 2025Attention to the Burstiness in Visual Prompt Tuning!cited by 4
  16. 2025Posterior Collapse as a Phase Transition in Variational Autoencoderscited by 4
  17. 2026Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Trainingcited by 4
  18. 2021Improving Generalization of Batch Whitening by Convolutional Unit Optimizationcited by 3
  19. 2021Unity is strength: Improving the Detection of Adversarial Examples with Ensemble Approachescited by 3
  20. 2022What can we Learn by Predicting Accuracy?cited by 3
  21. 2023Adaptive whitening in neural populations with gain-modulating interneuronscited by 3
  22. 2023Quaternion Convolutional Neural Networks: Current Advances and Future Directionscited by 3
  23. 20244DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splattingcited by 3
  24. 2026CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networkscited by 3
  25. 2022Spoofing-Aware Speaker Verification with Unsupervised Domain Adaptationcited by 2
  26. 2022Reduced-order modeling for parameterized large-eddy simulations of atmospheric pollutant dispersioncited by 2
  27. 2023SASSL: Enhancing Self-Supervised Learning via Neural Style Transfercited by 2
  28. 2024Multi-scale decomposition of sea surface height snapshots using machine learningcited by 2
  29. 2024Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Settingcited by 2
  30. 2024Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Searchcited by 2
  31. 2025Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Modelcited by 2
  32. 2025C3R: Channel Conditioned Cell Representations for unified evaluation in microscopy imagingcited by 2
  33. 2025Controllable Feature Whitening for Hyperparameter-Free Bias Mitigationcited by 2
  34. 2025BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compressioncited by 2
  35. 2026Preconditioned Flow Matchingcited by 2
  36. 2026One Adapter Pair per Model: A Universal Activation Interface for Language Modelscited by 2
  37. 2021Interpretable Face Manipulation Detection via Feature Whiteningcited by 1
  38. 2024Estimation and Deconvolution of Second Order Cyclostationary Signalscited by 1

50 most cited of 97 All 97 (JSON)