# MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction

arXiv: https://arxiv.org/abs/2410.23577 · HTML: https://mlsanity.com/paper/2410.23577/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 45 of its 88 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Focal Frequency Loss for Image Reconstruction and Synthesis](https://mlsanity.com/paper/2012.12821/) (2020)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Deep Learning for Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/1808.03344/) (2018)
- [Adaptive Diffusion Priors for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2207.05876/) (2022)
- [Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1712.01751/) (2017)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](https://mlsanity.com/paper/1709.00753/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction](https://mlsanity.com/paper/1703.00555/) (2017)
- [DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior](https://mlsanity.com/paper/2001.03799/) (2020)
- [Towards performant and reliable undersampled MR reconstruction via diffusion model sampling](https://mlsanity.com/paper/2203.04292/) (2022)
- [Reducing Uncertainty in Undersampled MRI Reconstruction with Active Acquisition](https://mlsanity.com/paper/1902.03051/) (2019)
- [Swin Transformer for Fast MRI](https://mlsanity.com/paper/2201.03230/) (2022)
- [High-Frequency Space Diffusion Models for Accelerated MRI](https://mlsanity.com/paper/2208.05481/) (2022)

## Abstract (arXiv)

Non-semantic context information is crucial for visual recognition, as the human visual perception system first uses global statistics to process scenes rapidly before identifying specific objects. However, while semantic information is increasingly incorporated into computer vision tasks such as image reconstruction, non-semantic information, such as global spatial structures, is often overlooked. To bridge the gap, we propose a biologically informed non-semantic context descriptor, MS-Glance, along with the Glance Index Measure for comparing two images. A Global Glance vector is formulated by randomly retrieving pixels based on a perception-driven rule from an image to form a vector representing non-semantic global context, while a local Glance vector is a flattened local image window, mimicking a zoom-in observation. The Glance Index is defined as the inner product of two standardized sets of Glance vectors. We evaluate the effectiveness of incorporating Glance supervision in two reconstruction tasks: image fitting with implicit neural representation (INR) and undersampled MRI reconstruction. Extensive experimental results show that MS-Glance outperforms existing image restoration losses across both natural and medical images. The code is available at urlhttps://github.com/Z7Gao/MSGlance.