mlsanityData to 14 Oct 2026

Phase-space entropy at acquisition reflects downstream learnability

arXiv 2512.19223 ↗First posted 22 Dec 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern learning systems work with data that vary widely across domains, but they all ultimately depend on how much structure is already present in the measurements before any model is trained. This raises a basic question: is there a general, modality-agnostic way to quantify how acquisition itself preserves or destroys the information that downstream learners could use? Here we propose an acquisition-level scalar Delta Smathcal B based on instrument-resolved phase space. Unlike pixelwise distortion or purely spectral errors that often saturate under aggressive undersampling, Delta Smathcal B directly quantifies how acquisition mixes or removes joint space--frequency structure at the instrument scale. We show theoretically that (Delta Smathcal B) correctly identifies the phase-space coherence of periodic sampling as the physical source of aliasing, recovering classical sampling-theorem consequences. Empirically, across masked image classification, accelerated MRI, and massive MIMO (including over-the-air measurements), |Delta Smathcal B| consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty without training. In particular, minimizing |Delta Smathcal B| enables zero-training selection of variable-density MRI mask parameters that matches designs tuned by conventional pre-reconstruction criteria. These results suggest that phase-space entropy at acquisition reflects downstream learnability, enabling pre-training selection of candidate sampling policies and as a shared notion of information preservation across modalities.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 49 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

16 of 49 references.

References · 16

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  8. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  9. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
  10. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  11. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 274
  12. 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 226
4 more
  1. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  2. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  3. 2022On the uncertainty principle of neural networkscited by 4
  4. 2024Quantum-Inspired Analysis of Neural Network Vulnerabilities: The Role of Conjugate Variables in System Attackscited by 4

Built on by

Not cited yet.