# COPA: Constrained PARAFAC2 for Sparse & Large Datasets

arXiv: https://arxiv.org/abs/1803.04572 · HTML: https://mlsanity.com/paper/1803.04572/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2018.

## Built on by (most cited first, 18 of 18)

- [ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://mlsanity.com/paper/2206.01861/) (2022)
- [ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization](https://mlsanity.com/paper/2406.05981/) (2024)
- [ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks](https://mlsanity.com/paper/2312.08583/) (2023)
- [DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing](https://mlsanity.com/paper/2212.03597/) (2022)
- [An AO-ADMM approach to constraining PARAFAC2 on all modes](https://mlsanity.com/paper/2110.01278/) (2021)
- [Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers](https://mlsanity.com/paper/2211.11586/) (2022)
- [DPar2: Fast and Scalable PARAFAC2 Decomposition for Irregular Dense Tensors](https://mlsanity.com/paper/2203.12798/) (2022)
- [Unsupervised EHR-based Phenotyping via Matrix and Tensor Decompositions](https://mlsanity.com/paper/2209.00322/) (2022)
- [D-Tracker: Modeling Interest Diffusion in Social Activity Tensor Data Streams](https://mlsanity.com/paper/2505.00242/) (2025)
- [Revisiting Large Language Model Pruning using Neuron Semantic Attribution](https://mlsanity.com/paper/2503.01542/) (2025)
- [PARAFAC2 AO-ADMM: Constraints in all modes](https://mlsanity.com/paper/2102.02087/) (2021)
- [MULTIPAR: Supervised Irregular Tensor Factorization with Multi-task Learning](https://mlsanity.com/paper/2208.00993/) (2022)
- [Fast and Accurate Dual-Way Streaming PARAFAC2 for Irregular Tensors -- Algorithm and Application](https://mlsanity.com/paper/2305.18376/) (2023)
- [SWoTTeD: An Extension of Tensor Decomposition to Temporal Phenotyping](https://mlsanity.com/paper/2310.01201/) (2023)
- [PARAFAC2-based Coupled Matrix and Tensor Factorizations with Constraints](https://mlsanity.com/paper/2406.12338/) (2024)
- [tPARAFAC2: Tracking evolving patterns in (incomplete) temporal data](https://mlsanity.com/paper/2407.01356/) (2024)
- [Toward Temporal Causal Representation Learning with Tensor Decomposition](https://mlsanity.com/paper/2507.14126/) (2025)
- [Modeling and Mining Multi-Aspect Graphs With Scalable Streaming Tensor Decomposition](https://mlsanity.com/paper/2210.04404/) (2022)

## Abstract (arXiv)

PARAFAC2 has demonstrated success in modeling irregular tensors, where the tensor dimensions vary across one of the modes. An example scenario is modeling treatments across a set of patients with the varying number of medical encounters over time. Despite recent improvements on unconstrained PARAFAC2, its model factors are usually dense and sensitive to noise which limits their interpretability. As a result, the following open challenges remain: a) various modeling constraints, such as temporal smoothness, sparsity and non-negativity, are needed to be imposed for interpretable temporal modeling and b) a scalable approach is required to support those constraints efficiently for large datasets. To tackle these challenges, we propose a it COnstrained it PARAFAC2 (COPA) method, which carefully incorporates optimization constraints such as temporal smoothness, sparsity, and non-negativity in the resulting factors. To efficiently support all those constraints, COPA adopts a hybrid optimization framework using alternating optimization and alternating direction method of multiplier (AO-ADMM). As evaluated on large electronic health record (EHR) datasets with hundreds of thousands of patients, COPA achieves significant speedups (up to 36 times faster) over prior PARAFAC2 approaches that only attempt to handle a subset of the constraints that COPA enables. Overall, our method outperforms all the baselines attempting to handle a subset of the constraints in terms of speed, while achieving the same level of accuracy. Through a case study on temporal phenotyping of medically complex children, we demonstrate how the constraints imposed by COPA reveal concise phenotypes and meaningful temporal profiles of patients. The clinical interpretation of both the phenotypes and the temporal profiles was confirmed by a medical expert.