# Personalized Dictionary Learning for Heterogeneous Datasets

arXiv: https://arxiv.org/abs/2305.15311 · HTML: https://mlsanity.com/paper/2305.15311/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 18 of 18)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis](https://mlsanity.com/paper/1704.00708/) (2017)
- [Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture](https://mlsanity.com/paper/1511.03607/) (2015)
- [Simple, Efficient, and Neural Algorithms for Sparse Coding](https://mlsanity.com/paper/1503.00778/) (2015)
- [Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization](https://mlsanity.com/paper/1310.7991/) (2013)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [New Algorithms for Learning Incoherent and Overcomplete Dictionaries](https://mlsanity.com/paper/1308.6273/) (2013)
- [GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning](https://mlsanity.com/paper/2108.02741/) (2021)
- [Personalized PCA: Decoupling Shared and Unique Features](https://mlsanity.com/paper/2207.08041/) (2022)
- [Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Rank-1 Robust Principal Component Analysis](https://mlsanity.com/paper/1812.11466/) (2018)
- [Simple Alternating Minimization Provably Solves Complete Dictionary Learning](https://mlsanity.com/paper/2210.12816/) (2022)
- [Analysis of Fast Structured Dictionary Learning](https://mlsanity.com/paper/1805.12529/) (2018)
- [Alternating minimization for dictionary learning: Local Convergence Guarantees](https://mlsanity.com/paper/1711.03634/) (2017)

## Built on by (most cited first, 5 of 5)

- [Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective](https://mlsanity.com/paper/2411.07523/) (2024)
- [Personalized Coupled Tensor Decomposition for Multimodal Data Fusion: Uniqueness and Algorithms](https://mlsanity.com/paper/2412.01102/) (2024)
- [Personalized Convolutional Dictionary Learning of Physiological Time Series](https://mlsanity.com/paper/2503.07687/) (2025)
- [Dataset Dictionary Learning in a Wasserstein Space for Federated Domain Adaptation](https://mlsanity.com/paper/2407.11647/) (2024)
- [Atom dimension adaptation for infinite set dictionary learning](https://mlsanity.com/paper/2409.06831/) (2024)

## Abstract (arXiv)

We introduce a relevant yet challenging problem named Personalized Dictionary Learning (PerDL), where the goal is to learn sparse linear representations from heterogeneous datasets that share some commonality. In PerDL, we model each dataset's shared and unique features as global and local dictionaries. Challenges for PerDL not only are inherited from classical dictionary learning (DL), but also arise due to the unknown nature of the shared and unique features. In this paper, we rigorously formulate this problem and provide conditions under which the global and local dictionaries can be provably disentangled. Under these conditions, we provide a meta-algorithm called Personalized Matching and Averaging (PerMA) that can recover both global and local dictionaries from heterogeneous datasets. PerMA is highly efficient; it converges to the ground truth at a linear rate under suitable conditions. Moreover, it automatically borrows strength from strong learners to improve the prediction of weak learners. As a general framework for extracting global and local dictionaries, we show the application of PerDL in different learning tasks, such as training with imbalanced datasets and video surveillance.