mlsanityData to 14 Oct 2026

Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysis

arXiv 2410.04703 ↗First posted 7 Oct 2024cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Neural time-series analysis has traditionally focused on modeling data in the time domain, often with some approaches incorporating equivalent Fourier domain representations as auxiliary spectral features. In this work, we shift the main focus to frequency representations, modeling time-series data fully and directly in the Fourier domain. We introduce Neural Fourier Modelling (NFM), a compact yet powerful solution for time-series analysis. NFM is grounded in two key properties of the Fourier transform (FT): (i) the ability to model finite-length time series as functions in the Fourier domain, treating them as continuous-time elements in function space, and (ii) the capacity for data manipulation (such as resampling and timespan extension) within the Fourier domain. We reinterpret Fourier-domain data manipulation as frequency extrapolation and interpolation, incorporating this as a core learning mechanism in NFM, applicable across various tasks. To support flexible frequency extension with spectral priors and effective modulation of frequency representations, we propose two learning modules: Learnable Frequency Tokens (LFT) and Implicit Neural Fourier Filters (INFF). These modules enable compact and expressive modeling in the Fourier domain. Extensive experiments demonstrate that NFM achieves state-of-the-art performance on a wide range of tasks (forecasting, anomaly detection, and classification), including challenging time-series scenarios with previously unseen sampling rates at test time. Moreover, NFM is highly compact, requiring fewer than 40K parameters in each task, with time-series lengths ranging from 100 to 16K.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 51 of its 61 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

51 of 61 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  11. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  12. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
38 more
  1. 2018On the Spectral Bias of Neural Networkscited by 1,341
  2. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  3. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  4. 2016HyperNetworkscited by 1,027
  5. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  8. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 552
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  10. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  11. 2021FNet: Mixing Tokens with Fourier Transformscited by 347
  12. 2021A Fourier-based Framework for Domain Generalizationcited by 290
  13. 2021Global Filter Networks for Image Classificationcited by 260
  14. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  15. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 248
  16. 2021Mobile-Former: Bridging MobileNet and Transformercited by 235
  17. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
  18. 2017Deep Complex Networkscited by 217
  19. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 206
  20. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  21. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 172
  22. 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
  23. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 144
  24. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
  25. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 127
  26. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 120
  27. 2021A Structured Dictionary Perspective on Implicit Neural Representationscited by 100
  28. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  29. 2019Phase-aware Speech Enhancement with Deep Complex U-Netcited by 73
  30. 2022Masked Frequency Modeling for Self-Supervised Visual Pre-Trainingcited by 71
  31. 2019Latent Normalizing Flows for Discrete Sequencescited by 67
  32. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  33. 2020What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encodingcited by 52
  34. 2023Adaptive Frequency Filters As Efficient Global Token Mixerscited by 45
  35. 2022Deep Frequency Filtering for Domain Generalizationcited by 43
  36. 2022Generalizable Implicit Neural Representations via Instance Pattern Composerscited by 39
  37. 2020Modeling Continuous Stochastic Processes with Dynamic Normalizing Flowscited by 37
  38. 2023Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokenscited by 35

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.