mlsanityData to 14 Oct 2026

From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion

arXiv 2606.12303 ↗First posted 10 Jun 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

34 of 49 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  7. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  8. 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 317
  9. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  10. 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 223
  11. 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
  12. 2023DDFM: Denoising Diffusion Model for Multi-Modality Image Fusioncited by 153
22 more
  1. 2023Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentationcited by 129
  2. 2025FlexTok: Resampling Images into 1D Token Sequences of Flexible Lengthcited by 110
  3. 2023Equivariant Multi-Modality Image Fusioncited by 99
  4. 2023LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Imagescited by 87
  5. 2024Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusioncited by 84
  6. 2020DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusioncited by 73
  7. 2024Task-Customized Mixture of Adapters for General Image Fusioncited by 59
  8. 2025Infrared and Visible Image Fusion: From Data Compatibility to Task Adaptioncited by 46
  9. 2024MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusioncited by 35
  10. 2025Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyondcited by 35
  11. 2025DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusioncited by 34
  12. 2024Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Modelcited by 31
  13. 2024Task-driven Image Fusion with Learnable Fusion Losscited by 25
  14. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  15. 2023From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusioncited by 20
  16. 2025Highly Compressed Tokenizer Can Generate Without Trainingcited by 18
  17. 2025Efficient Rectified Flow for Image Fusioncited by 15
  18. 2025Hita: Holistic Tokenizer for Autoregressive Image Generationcited by 11
  19. 2026UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservationcited by 6
  20. 2026ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generationcited by 5
  21. 2022TransFuse: A Unified Transformer-based Image Fusion Framework using Self-supervised Learningcited by 4
  22. 2026Bridging Human Evaluation to Infrared and Visible Image Fusioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.