# Symphony in the Latent Space: Provably Integrating High-dimensional Techniques with Non-linear Machine Learning Models

arXiv: https://arxiv.org/abs/2212.00852 · HTML: https://mlsanity.com/paper/2212.00852/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 81 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 26 of 26)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis](https://mlsanity.com/paper/1704.00708/) (2017)
- [Temporal Relational Ranking for Stock Prediction](https://mlsanity.com/paper/1809.09441/) (2018)
- [Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction](https://mlsanity.com/paper/1712.02136/) (2017)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [Deep Learning in Asset Pricing](https://mlsanity.com/paper/1904.00745/) (2019)
- [Dropping Convexity for Faster Semi-definite Optimization](https://mlsanity.com/paper/1509.03917/) (2015)
- [AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks](https://mlsanity.com/paper/1908.02646/) (2019)
- [Approximation beats concentration? An approximation view on inference with smooth radial kernels](https://mlsanity.com/paper/1801.03437/) (2018)
- [More Algorithms for Provable Dictionary Learning](https://mlsanity.com/paper/1401.0579/) (2014)
- [On Exact Learning Monotone DNF from Membership Queries](https://mlsanity.com/paper/1405.0792/) (2014)
- [Equity2Vec: End-to-end Deep Learning Framework for Cross-sectional Asset Pricing](https://mlsanity.com/paper/1909.04497/) (2019)
- [Stock Market Prediction from WSJ: Text Mining via Sparse Matrix Factorization](https://mlsanity.com/paper/1406.7330/) (2014)
- [Properties of Latent Variable Network Models](https://mlsanity.com/paper/1506.07806/) (2015)
- [BATS: A Spectral Biclustering Approach to Single Document Topic Modeling and Segmentation](https://mlsanity.com/paper/2008.02218/) (2020)
- [Adaptive Reduced Rank Regression](https://mlsanity.com/paper/1905.11566/) (2019)
- [AA-Forecast: Anomaly-Aware Forecast for Extreme Events](https://mlsanity.com/paper/2208.09933/) (2022)
- [From which world is your graph?](https://mlsanity.com/paper/1711.00982/) (2017)
- [Learning Personalized Representations using Graph Convolutional Network](https://mlsanity.com/paper/2207.14298/) (2022)

## Built on by (most cited first, 1 of 1)

- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)

## Abstract (arXiv)

This paper revisits building machine learning algorithms that involve interactions between entities, such as those between financial assets in an actively managed portfolio, or interactions between users in a social network. Our goal is to forecast the future evolution of ensembles of multivariate time series in such applications (e.g., the future return of a financial asset or the future popularity of a Twitter account). Designing ML algorithms for such systems requires addressing the challenges of high-dimensional interactions and non-linearity. Existing approaches usually adopt an ad-hoc approach to integrating high-dimensional techniques into non-linear models and recent studies have shown these approaches have questionable efficacy in time-evolving interacting systems. To this end, we propose a novel framework, which we dub as the additive influence model. Under our modeling assumption, we show that it is possible to decouple the learning of high-dimensional interactions from the learning of non-linear feature interactions. To learn the high-dimensional interactions, we leverage kernel-based techniques, with provable guarantees, to embed the entities in a low-dimensional latent space. To learn the non-linear feature-response interactions, we generalize prominent machine learning techniques, including designing a new statistically sound non-parametric method and an ensemble learning algorithm optimized for vector regressions. Extensive experiments on two common applications demonstrate that our new algorithms deliver significantly stronger forecasting power compared to standard and recently proposed methods.