# A Tutorial on Principal Component Analysis

arXiv: https://arxiv.org/abs/1404.1100 · HTML: https://mlsanity.com/paper/1404.1100/

## Key facts

- Cited by 141 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2014.

## Built on by (most cited first, 30 of 141)

- [Flora: Low-Rank Adapters Are Secretly Gradient Compressors](https://mlsanity.com/paper/2402.03293/) (2024)
- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [RudolfV: A Foundation Model by Pathologists for Pathologists](https://mlsanity.com/paper/2401.04079/) (2024)
- [Machine Learning in Aerodynamic Shape Optimization](https://mlsanity.com/paper/2202.07141/) (2022)
- [Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell](https://mlsanity.com/paper/2406.14673/) (2024)
- [Gradient-Free Textual Inversion](https://mlsanity.com/paper/2304.05818/) (2023)
- ["Principal Components" Enable A New Language of Images](https://mlsanity.com/paper/2503.08685/) (2025)
- [Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation](https://mlsanity.com/paper/2311.13209/) (2023)
- [FlowTS: Time Series Generation via Rectified Flow](https://mlsanity.com/paper/2411.07506/) (2024)
- [coVariance Neural Networks](https://mlsanity.com/paper/2205.15856/) (2022)
- [Generative Recursive Reasoning](https://mlsanity.com/paper/2605.19376/) (2026)
- [Fundamentals of Task-Agnostic Data Valuation](https://mlsanity.com/paper/2208.12354/) (2022)
- [Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset](https://mlsanity.com/paper/2306.11167/) (2023)
- [CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration](https://mlsanity.com/paper/2309.07822/) (2023)
- [MuseCL: Predicting Urban Socioeconomic Indicators via Multi-Semantic Contrastive Learning](https://mlsanity.com/paper/2407.09523/) (2024)
- [AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation](https://mlsanity.com/paper/2501.07810/) (2025)
- [Empirical comparison between autoencoders and traditional dimensionality reduction methods](https://mlsanity.com/paper/2103.04874/) (2021)
- [Uncertainty Aware Learning for Language Model Alignment](https://mlsanity.com/paper/2406.04854/) (2024)
- [Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization](https://mlsanity.com/paper/2408.10672/) (2024)
- [Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing](https://mlsanity.com/paper/2510.01243/) (2025)
- [Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models](https://mlsanity.com/paper/2206.14615/) (2022)
- [A Unified Multimodal De- and Re-coupling Framework for RGB-D Motion Recognition](https://mlsanity.com/paper/2211.09146/) (2022)
- [Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning](https://mlsanity.com/paper/2404.14607/) (2024)
- [Optimal training of finitely-sampled quantum reservoir computers for forecasting of chaotic dynamics](https://mlsanity.com/paper/2409.01394/) (2024)
- [CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges](https://mlsanity.com/paper/2603.11863/) (2026)
- [Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering](https://mlsanity.com/paper/2201.05077/) (2022)
- [SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models](https://mlsanity.com/paper/2508.03351/) (2025)
- [Sparse Covariance Neural Networks](https://mlsanity.com/paper/2410.01669/) (2024)
- [Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks](https://mlsanity.com/paper/2305.01713/) (2023)
- [DReSD: Dense Retrieval for Speculative Decoding](https://mlsanity.com/paper/2502.15572/) (2025)

## Abstract (arXiv)

Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but (sometimes) poorly understood. The goal of this paper is to dispel the magic behind this black box. This manuscript focuses on building a solid intuition for how and why principal component analysis works. This manuscript crystallizes this knowledge by deriving from simple intuitions, the mathematics behind PCA. This tutorial does not shy away from explaining the ideas informally, nor does it shy away from the mathematics. The hope is that by addressing both aspects, readers of all levels will be able to gain a better understanding of PCA as well as the when, the how and the why of applying this technique.