# Frequency Spectrum is More Effective for Multimodal Representation and Fusion: A Multimodal Spectrum Rumor Detector

arXiv: https://arxiv.org/abs/2312.11023 · HTML: https://mlsanity.com/paper/2312.11023/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [Global Filter Networks for Image Classification](https://mlsanity.com/paper/2107.00645/) (2021)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [Learning in the Frequency Domain](https://mlsanity.com/paper/2002.12416/) (2020)
- [FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective](https://mlsanity.com/paper/2311.06190/) (2023)
- [Connecting the Dots: Identifying Network Structure via Graph Signal Processing](https://mlsanity.com/paper/1810.13066/) (2018)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction](https://mlsanity.com/paper/2004.00574/) (2020)
- [Deep Residual Learning in the JPEG Transform Domain](https://mlsanity.com/paper/1812.11690/) (2018)
- [FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions](https://mlsanity.com/paper/1811.08257/) (2018)
- [Interpretable Multimodal Misinformation Detection with Logic Reasoning](https://mlsanity.com/paper/2305.05964/) (2023)

## Built on by (most cited first, 9 of 9)

- [VGA: Vision and Graph Fused Attention Network for Rumor Detection](https://mlsanity.com/paper/2401.01759/) (2024)
- [CoSD: Collaborative Stance Detection with Contrastive Heterogeneous Topic Graph Learning](https://mlsanity.com/paper/2404.17609/) (2024)
- [Dynamic Analysis and Adaptive Discriminator for Fake News Detection](https://mlsanity.com/paper/2408.10883/) (2024)
- [Q-FSRU: Quantum-Augmented Frequency-Spectral Fusion for Medical Visual Question Answering](https://mlsanity.com/paper/2508.12036/) (2025)
- [Q-FSRU: Quantum-Augmented Frequency-Spectral For Medical Visual Question Answering](https://mlsanity.com/paper/2509.23899/) (2025)
- [Bridging Pixels and Words: Mask-Aware Local Semantic Fusion for Multimodal Media Verification](https://mlsanity.com/paper/2603.26052/) (2026)
- [Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation](https://mlsanity.com/paper/2607.26555/) (2026)
- [Frequency-Domain Dual-Branch Fusion for Medical Visual Question Answering](https://mlsanity.com/paper/2608.08307/) (2026)
- [The Hidden States Cookbook: A Large-Scale Ablation Study for Noise-Robust Conversational Intent Classification in Industry](https://mlsanity.com/paper/2610.05394/) (2026)

## Abstract (arXiv)

Multimodal content, such as mixing text with images, presents significant challenges to rumor detection in social media. Existing multimodal rumor detection has focused on mixing tokens among spatial and sequential locations for unimodal representation or fusing clues of rumor veracity across modalities. However, they suffer from less discriminative unimodal representation and are vulnerable to intricate location dependencies in the time-consuming fusion of spatial and sequential tokens. This work makes the first attempt at multimodal rumor detection in the frequency domain, which efficiently transforms spatial features into the frequency spectrum and obtains highly discriminative spectrum features for multimodal representation and fusion. A novel Frequency Spectrum Representation and fUsion network (FSRU) with dual contrastive learning reveals the frequency spectrum is more effective for multimodal representation and fusion, extracting the informative components for rumor detection. FSRU involves three novel mechanisms: utilizing the Fourier transform to convert features in the spatial domain to the frequency domain, the unimodal spectrum compression, and the cross-modal spectrum co-selection module in the frequency domain. Substantial experiments show that FSRU achieves satisfactory multimodal rumor detection performance.