mlsanityData to 14 Oct 2026

STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction

arXiv 2312.17346 ↗First posted 28 Dec 2023cs.LG cs.AI cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learn temporal representation and cross-series representation using two tandem sparse Hopfield layers. In addition, StanHop incorporates two additional external memory modules: a Plug-and-Play module and a Tune-and-Play module for train-less and task-aware memory-enhancements, respectively. They allow StanHop-Net to swiftly respond to certain sudden events. Methodologically, we construct the StanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a sparse extension of the modern Hopfield model (Generalized Sparse Modern Hopfield Model) and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of our framework on both synthetic and real-world settings.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 29 of its 62 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

29 of 62 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  6. 2020Reformer: The Efficient Transformercited by 1,668
  7. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  8. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  9. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  10. 2014Neural Turing Machinescited by 644
  11. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  12. 2018Deep learning for time series classification: a reviewcited by 576
17 more
  1. 2020Hopfield Networks is All You Needcited by 475
  2. 2014Memory Networkscited by 363
  3. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  4. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  5. 2019Sparse Sequence-to-Sequence Modelscited by 170
  6. 2019Adaptively Sparse Transformerscited by 155
  7. 2019Learning with Fenchel-Young Lossescited by 119
  8. 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 112
  9. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 89
  10. 2023Energy Transformercited by 71
  11. 2017Learning to Remember Rare Eventscited by 63
  12. 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 55
  13. 2020Modern Hopfield Networks and Attention for Immune Repertoire Classificationcited by 53
  14. 2020Machine Learning Advances for Time Series Forecastingcited by 49
  15. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  16. 2023Context-enriched molecule representations improve few-shot drug discoverycited by 24
  17. 2023Feature Programming for Multivariate Time Series Predictioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 24; 2025: 8; 2026: 3.

35 papers.

Sort:

By area:

  1. 2024Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Modelscited by 38
  2. 2024Outlier-Efficient Hopfield Layers for Large Transformer-Based Modelscited by 38
  3. 2024Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicatorscited by 32
  4. 2024Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Modelscited by 31
  5. 2024The Fine-Grained Complexity of Gradient Computation for Training Large Language Modelscited by 30
  6. 2024Circuit Complexity Bounds for RoPE-based Transformer Architecturecited by 26
  7. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  8. 2024Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descentcited by 22
  9. 2024BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Modelcited by 21
  10. 2024The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexitycited by 21
  11. 2024A Tighter Complexity Analysis of SparseGPTcited by 19
  12. 2024HSR-Enhanced Sparse Attention Accelerationcited by 19
23 more
  1. 2024Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codescited by 19
  2. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 15
  3. 2024Provable Differentially Private Computation of the Cross-Attention Mechanismcited by 14
  4. 2024Theoretical Constraints on the Expressive Power of mathsfRoPE-based Tensor Attention Transformerscited by 12
  5. 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flowscited by 10
  6. 2024Decoupled Alignment for Robust Plug-and-Play Adaptationcited by 9
  7. 2024Modern Hopfield Networks Require Chain-of-Thought to Solve mathsfNC1-Hard Problemscited by 9
  8. 2025Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transformcited by 9
  9. 2024Dense Associative Memory Through the Lens of Random Featurescited by 8
  10. 2025PLADIS: Pushing the Limits of Attention in Diffusion Models at Inference Time by Leveraging Sparsitycited by 7
  11. 2024Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Studycited by 4
  12. 2024Unlocking the Theory Behind Scaling 1-Bit Neural Networkscited by 4
  13. 2025Neural Algorithmic Reasoning for Hypergraphs with Looped Transformerscited by 4
  14. 2025Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approachescited by 4
  15. 2024CoRMF: Criticality-Ordered Recurrent Mean Field Ising Solvercited by 3
  16. 2024Differentially Private Kernel Density Estimationcited by 3
  17. 2025Transformers as Intrinsic Optimizers: Forward Inference through the Energy Principlecited by 3
  18. 2025Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decodingcited by 2
  19. 2025Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiencycited by 2
  20. 2026Sparse Attention as Compact Kernel Regressioncited by 2
  21. 2024Predictive Attractor Modelscited by 0
  22. 2026Geometry-Aware Attention Guidance for Diffusion Models via Modern Hopfield Dynamicscited by 0
  23. 2026Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Trackingcited by 0