mlsanityData to 14 Oct 2026

Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputs

arXiv 2505.00473 ↗First posted 1 May 2025cs.LG cs.NA math.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

We explore the promising performance of a transformer model in predicting outputs of parametric dynamical systems with external time-varying input signals. The outputs of such systems vary not only with physical parameters but also with external time-varying input signals. Accurately catching the dynamics of such systems is challenging. We have adapted and extended an existing transformer model, called temporal fusion transformer (TFT), for single-output prediction to a multiple-output transformer, named as Spatially-Enhanced Temporal Fusion Transformer (SE-TFT), which is able to predict multiple output responses of these systems. The SE-TFT generalizes the interpretability of the original TFT model. The generalized interpretable attention weight matrix explores not only the temporal correlations in the sequence, but also the interactions between the multiple outputs, providing explanation for the spatial correlation in the output domain. This proposed SE-TFT accurately predicts the sequence of multiple outputs, regardless of the nonlinearity of the system and the dimensionality of the parameter space.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

28 of 53 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  5. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  6. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  7. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  8. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  9. 2022Transformers in Time Series: A Surveycited by 497
  10. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  11. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 336
  12. 2020Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Casecited by 129
16 more
  1. 2020Transformers for Modeling Physical Systemscited by 106
  2. 2020A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEscited by 105
  3. 2021POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decompositioncited by 79
  4. 2023β-Variational autoencoders and transformers for reduced-order modelling of fluid flowscited by 77
  5. 2022Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Versioncited by 59
  6. 2022Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecastingcited by 54
  7. 2022TACTiS: Transformer-Attentional Copulas for Time Seriescited by 38
  8. 2022Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reductioncited by 30
  9. 2022gLaSDI: Parametric Physics-informed Greedy Latent Space Dynamics Identificationcited by 30
  10. 2023GPLaSDI: Gaussian Process-based Interpretable Latent Space Dynamics Identification through Deep Autoencodercited by 30
  11. 2022Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutionscited by 29
  12. 2022Multi-fidelity surrogate modeling using long short-term memory networkscited by 18
  13. 2021SSDNet: State Space Decomposition Neural Network for Time Series Forecastingcited by 15
  14. 2020A greedy non-intrusive reduced order model for shallow water equationscited by 12
  15. 2021TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecastingcited by 8
  16. 2024Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identificationcited by 0