# Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution arXiv: https://arxiv.org/abs/2401.00740 ยท HTML: https://mlsanity.com/paper/2401.00740/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 84 references that are in the map. - First posted on arXiv 1 Jan 2024. ## Builds on (most cited first, 30 of 32) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016) - [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021) - [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017) - [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018) - [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015) - [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020) - [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018) - [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022) - [Conditional Positional Encodings for Vision Transformers](https://mlsanity.com/paper/2102.10882/) (2021) - [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](https://mlsanity.com/paper/2311.16518/) (2023) - [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016) - [Efficient Long-Range Attention Network for Image Super-resolution](https://mlsanity.com/paper/2203.06697/) (2022) - [Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2108.11084/) (2021) - [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020) - [Disentangling Light Fields for Super-Resolution and Disparity Estimation](https://mlsanity.com/paper/2202.10603/) (2022) - [Spatial-Angular Interaction for Light Field Image Super-Resolution](https://mlsanity.com/paper/1912.07849/) (2019) - [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020) - [Light Field Image Super-Resolution with Transformers](https://mlsanity.com/paper/2108.07597/) (2021) - [A 4D Light-Field Dataset and CNN Architectures for Material Recognition](https://mlsanity.com/paper/1608.06985/) (2016) - [Detail-Preserving Transformer for Light Field Image Super-Resolution](https://mlsanity.com/paper/2201.00346/) (2022) - [Occlusion-Aware Cost Constructor for Light Field Depth Estimation](https://mlsanity.com/paper/2203.01576/) (2022) - [Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution](https://mlsanity.com/paper/2302.08058/) (2023) - [Light Field Super-Resolution Via Graph-Based Regularization](https://mlsanity.com/paper/1701.02141/) (2017) - [Learning Sub-Pixel Disparity Distribution for Light Field Depth Estimation](https://mlsanity.com/paper/2208.09688/) (2022) - [Spatial-Angular Attention Network for Light Field Reconstruction](https://mlsanity.com/paper/2007.02252/) (2020) - [Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks](https://mlsanity.com/paper/1801.04314/) (2018) - [Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels](https://mlsanity.com/paper/2111.04069/) (2021) ## Built on by (most cited first, 3 of 3) - [L2FMamba: Lightweight Light Field Image Super-Resolution with State Space Model](https://mlsanity.com/paper/2503.19253/) (2025) - [RASLF: Representation-Aware State Space Model for Light Field Super-Resolution](https://mlsanity.com/paper/2603.16243/) (2026) - [VSANet: View-aware Sparse Attention Network for Light Field Image Denoising](https://mlsanity.com/paper/2606.24737/) (2026) ## Abstract (arXiv) The effective extraction of spatial-angular features plays a crucial role in light field image super-resolution (LFSR) tasks, and the introduction of convolution and Transformers leads to significant improvement in this area. Nevertheless, due to the large 4D data volume of light field images, many existing methods opted to decompose the data into a number of lower-dimensional subspaces and perform Transformers in each sub-space individually. As a side effect, these methods inadvertently restrict the self-attention mechanisms to a One-to-One scheme accessing only a limited subset of LF data, explicitly preventing comprehensive optimization on all spatial and angular cues. In this paper, we identify this limitation as subspace isolation and introduce a novel Many-to-Many Transformer (M2MT) to address it. M2MT aggregates angular information in the spatial subspace before performing the self-attention mechanism. It enables complete access to all information across all sub-aperture images (SAIs) in a light field image. Consequently, M2MT is enabled to comprehensively capture long-range correlation dependencies. With M2MT as the foundational component, we develop a simple yet effective M2MT network for LFSR. Our experimental results demonstrate that M2MT achieves state-of-the-art performance across various public datasets, and it offers a favorable balance between model performance and efficiency, yielding higher-quality LFSR results with substantially lower demand for memory and computation. We further conduct in-depth analysis using local attribution maps (LAM) to obtain visual interpretability, and the results validate that M2MT is empowered with a truly non-local context in both spatial and angular subspaces to mitigate subspace isolation and acquire effective spatial-angular representation.