# Dynamics-Based Intrinsic Signal Model for High-Dimensional, Small-Sample Data

arXiv: https://arxiv.org/abs/2304.06522 · HTML: https://mlsanity.com/paper/2304.06522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 6 of 6)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [AI Poincar'e: Machine Learning Conservation Laws from Trajectories](https://mlsanity.com/paper/2011.04698/) (2020)

## Abstract (arXiv)

Signal extraction is difficult when the number of variables N is much larger than the number of observations M. We address this problem under the working hypothesis that an empirical dataset consists of states sampled from underlying multivariate dynamics. Instead of treating the M observations as points in an N-dimensional variable space, we treat the N variables as points in an M-dimensional sample-coordinate space, interpreted as an effective time-delay coordinate space of the latent dynamics. This representation allows a large N to provide many points for estimating the variable distribution even when M is small. Transposed singular value decomposition (SVD) and the unsupervised feature-selection method of Taguchi are used to extract variable-side deviations from an estimated Gaussian background as signal candidates. As M is reduced, the effective separation between sampled states increases; contributions from finite-correlation components are expected to decay, whereas sufficiently long-correlation components can persist toward the small-sample limit. We define these persistent components as intrinsic signals and estimate them by extrapolation toward M=0. We first tested the method on high-dimensional, small-sample data explicitly generated by a randomised coupling strength globally coupled map (RCS-GCM), for which the long- and short-correlation components were known. The extracted intrinsic signals corresponded to the known long-correlation variables. We then applied the method to The Cancer Genome Atlas (TCGA) pan-kidney gene-expression data, which are not ordinarily treated as data explicitly generated by a dynamical system. Using an SVD component associated with the pathologic-M category, variable-side signal components were extracted from the 20,531-dimensional data even under small-sample conditions.