# The Lift Spectrum: How Measurement-to-Space Adaptivity Shapes Robustness in Image-Free Single-Pixel Sensing

arXiv: https://arxiv.org/abs/2607.22077 · HTML: https://mlsanity.com/paper/2607.22077/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 68 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Builds on (most cited first, 26 of 26)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [On hallucinations in tomographic image reconstruction](https://mlsanity.com/paper/2012.00646/) (2020)
- [Connecting Image Denoising and High-Level Vision Tasks via Deep Learning](https://mlsanity.com/paper/1809.01826/) (2018)
- [Joint CS-MRI Reconstruction and Segmentation with a Unified Deep Network](https://mlsanity.com/paper/1805.02165/) (2018)
- [Task-Aware Compressed Sensing with Generative Adversarial Networks](https://mlsanity.com/paper/1802.01284/) (2018)
- [TransCL: Transformer Makes Strong and Flexible Compressive Learning](https://mlsanity.com/paper/2207.11972/) (2022)
- [Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging](https://mlsanity.com/paper/2505.23180/) (2025)
- [Reconstruction-free action inference from compressive imagers](https://mlsanity.com/paper/1501.04367/) (2015)
- [Dual-Scale Transformer for Large-Scale Single-Pixel Imaging](https://mlsanity.com/paper/2404.05001/) (2024)
- [Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image](https://mlsanity.com/paper/1908.07623/) (2019)
- [Image-free single-pixel segmentation](https://mlsanity.com/paper/2108.10617/) (2021)

## Abstract (arXiv)

Single-pixel sensing encodes a scene as a short sequence of coded measurements, and image-free methods infer the task directly from that sequence. We show that removing image reconstruction relocates the central design problem to the lift: how 1D measurements become a 2D task representation. We organize this choice as a lift spectrum from a fixed-physics inverse, through a learned static projection, to content-adaptive retrieval. These are not interchangeable forms of reconstruction: the fixed-physics route reconstructs an image consumed at inference, whereas our spatiotemporal soft-fusion (STSF) network lifts measurements directly into task features, and task-prioritized loss scheduling (TPLS) uses a separate learned reconstruction branch only as scheduled training supervision. A probe-selected recurrent encoder and a parameter-matched lift ablation identify the STSF design. In simulation, STSF+TPLS exceeds the prior image-free baseline on three datasets at 3.13% sampling (+3.2 to +9.9 pp foreground mIoU) and remains competitive down to 0.39%. The strongest clean-trained reconstruct-then-segment baseline wins without measurement noise, but measurement noise reverses the ranking: the reconstructed task input carries a 20-70x larger normalized relative perturbation than the measurements themselves. Stressed to failure, the three lift regions exhibit distinct dominant signatures--collapse, imprinting, and coarsening. STSF+TPLS transfers without fine-tuning to a real single-pixel bench, where the reversal reappears as a proof of concept; inference takes about 14 ms per mask on an RTX 4090. Within the tested fixed-acquisition regime, measurement-to-space adaptivity therefore organizes both the clean-to-noisy operating envelope and the failure a system encounters. Code and pretrained weights: https://github.com/Hanyuyuan6/STSF-TPLS.