mlsanityData to 14 Oct 2026

Towards Gradient-based Time-Series Explanations through a SpatioTemporal Attention Network

arXiv 2405.17444 ↗First posted 18 May 2024cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore the feasibility of using a transformer-based, spatiotemporal attention network (STAN) for gradient-based time-series explanations. First, we trained the STAN model for video classifications using the global and local views of data and weakly supervised labels on time-series data (i.e. the type of an activity). We then leveraged a gradient-based XAI technique (e.g. saliency map) to identify salient frames of time-series data. According to the experiments using the datasets of four medically relevant activities, the STAN model demonstrated its potential to identify important frames of videos.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 18 May 2024.

Builds on

25 of 36 references.

References · 25

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2016Layer Normalizationcited by 5,669
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2017Decoupled Weight Decay Regularizationcited by 1,831
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,109
13 more
  1. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  2. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
  3. 2021Multiscale Vision Transformerscited by 897
  4. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  5. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  6. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  7. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
  8. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 184
  9. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  10. 2019Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Makingcited by 109
  11. 2020Benchmarking Deep Learning Interpretability in Time Series Predictionscited by 102
  12. 2021Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Surveycited by 72
  13. 2023Exploring a Gradient-based Explainable AI Technique for Time-Series Data: A Case Study of Assessing Stroke Rehabilitation Exercisescited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Extending Information Bottleneck Attribution to Video Sequencescited by 1
  2. 2024Interactive Example-based Explanations to Improve Health Professionals' Onboarding with AI for Human-AI Collaborative Decision Makingcited by 0
  3. 2025A Multimodal Fusion Model Leveraging MLP Mixer and Handcrafted Features-based Deep Learning Networks for Facial Palsy Detectioncited by 0