# Zoom and Shift are All You Need

arXiv: https://arxiv.org/abs/2406.08866 · HTML: https://mlsanity.com/paper/2406.08866/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Unifying Vision-and-Language Tasks via Text Generation](https://mlsanity.com/paper/2102.02779/) (2021)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Efficient Low-rank Multimodal Fusion with Modality-Specific Factors](https://mlsanity.com/paper/1806.00064/) (2018)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021)
- [Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrieval](https://mlsanity.com/paper/2204.09868/) (2022)
- [Deep Multimodal Learning for Audio-Visual Speech Recognition](https://mlsanity.com/paper/1501.05396/) (2015)
- [Modality-specific Cross-modal Similarity Measurement with Recurrent Attention Network](https://mlsanity.com/paper/1708.04776/) (2017)
- [A Personalized Zero-Shot ECG Arrhythmia Monitoring System: From Sparse Representation Based Domain Adaption to Energy Efficient Abnormal Beat Detection for Practical ECG Surveillance](https://mlsanity.com/paper/2207.07089/) (2022)

## Built on by (most cited first, 5 of 5)

- [Learning Domain-Invariant Representations for Cross-Domain Image Registration via Scene-Appearance Disentanglement](https://mlsanity.com/paper/2601.08875/) (2026)
- [Hierarchical Adaptive Expert for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2503.22715/) (2025)
- [SAS-Net: Cross-Domain Image Registration as Inverse Rendering via Structure-Appearance Factorization](https://mlsanity.com/paper/2602.09050/) (2026)
- [PCReg-Net: Progressive Contrast-Guided Registration for Cross-Domain Image Alignment](https://mlsanity.com/paper/2602.13304/) (2026)
- [Deformation-Free Cross-Domain Image Registration via Position-Encoded Temporal Attention](https://mlsanity.com/paper/2602.15959/) (2026)

## Abstract (arXiv)

Feature alignment serves as the primary mechanism for fusing multimodal data. We put forth a feature alignment approach that achieves full integration of multimodal information. This is accomplished via an alternating process of shifting and expanding feature representations across modalities to obtain a consistent unified representation in a joint feature space. The proposed technique can reliably capture high-level interplay between features originating from distinct modalities. Consequently, substantial gains in multimodal learning performance are attained. Additionally, we demonstrate the superiority of our approach over other prevalent multimodal fusion schemes on a range of tasks. Extensive experimental evaluation conducted on multimodal datasets comprising time series, image, and text demonstrates that our method achieves state-of-the-art results.