# On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problems

arXiv: https://arxiv.org/abs/2607.28080 · HTML: https://mlsanity.com/paper/2607.28080/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 8 of 8)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization](https://mlsanity.com/paper/2001.06270/) (2020)
- [A Practical Probabilistic Benchmark for AI Weather Models](https://mlsanity.com/paper/2401.15305/) (2024)
- [Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional data](https://mlsanity.com/paper/2401.01100/) (2024)
- [An entropy-optimal path to humble AI](https://mlsanity.com/paper/2506.17940/) (2025)
- [Fast, close, non-singular and property-preserving approximations of entropic measures](https://mlsanity.com/paper/2505.14234/) (2025)
- [Distillation and Interpretability of Ensemble Forecasts of ENSO Phase using Entropic Learning](https://mlsanity.com/paper/2602.16857/) (2026)

## Abstract (arXiv)

We extend a recently introduced Entropy-Optimal Manifold Clustering (EOMC) to allow for a joint simultaneous identification of subsets and subspaces of relevant features in nonstationary and nonlinear regression problems. It is shown that the proposed extension - that we coin as Entropy-Optimal Manifold Regression (EOMR) - allows a robust learning with linearly-scaling iteration and memory complexities. EOMR is compared to the most complete set of state-of-the-art tools from the Artificial Intelligence (AI) and Machine Learning (ML) that is available to the author, on the very challenging problems from chaotic and fluid dynamics: (i) on predicting the Lorenz-96 systems dynamics in strongly- and very-strongly chaotic regimes (with forcing parameter being F=8 and F=12, respectively); and, (ii) on a data from the Hasegawa-Wakatani model on the edge of the tokamak plasma. It is demonstrated that the proposed benchmarks (i) and (ii), indeed, are the very challenging problems for the state of the art ML and AI tools - since both the general-purpose gradient boosted random forests and deep neuronal networks, as well as transformer-based AI tools like TabPFN v.03 (more spezialised for large-dimensional small data learning problems) - result in orders of magnitude inferior root mean squared prediction errors, and orders of magnitude larger model complexities, when compared to the EOMR. For a Hasegawa-Wakatani example, EOMR distills a very simple entropy-optimal and skilful description of the leading Essential Orthogonal Function (EOF) dynamics, given by linear, causal and weakly-stationary autoregressive process described by just 8 parameters.