mlsanityData to 14 Oct 2026

Repurposing Foundation Model for Generalizable Medical Time Series Classification

arXiv 2410.03794 ↗First posted 3 Oct 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Medical time series (MedTS) classification suffers from poor generalizability in real-world deployment due to inter- and intra-dataset heterogeneity, such as varying numbers of channels, signal lengths, task definitions, and patient characteristics. To address this, we propose FORMED, a novel framework for repurposing a backbone foundation model, pre-trained on generic time series, to enable highly generalizable MedTS classification on unseen datasets. FORMED combines the backbone with a novel classifier comprising two components: (1) task-specific channel embeddings and label queries, dynamically sized to match any number of channels and target classes, and (2) a shared decoding attention layer, jointly trained across datasets to capture medical domain knowledge through task-agnostic feature-query interactions. After repurposing, FORMED achieves seamless adaptation to unseen MedTS datasets through lightweight label query training (0.1% of parameters), eliminating the need for full fine-tuning or architectural redesign. We evaluate FORMED on 5 diverse MedTS datasets, benchmarking against 11 Task-Specific Models (TSM) and 4 Task-Specific Adaptation (TSA) methods. Our results demonstrate FORMED's dominant performance, achieving up to 35% absolute improvement in F1-score (on ADFTD dataset) over specialized baselines. Further analysis reveals consistent generalization across varying channel configurations, time series lengths, and clinical tasks, which are key challenges in real-world deployment. By decoupling domain-invariant representation learning from task-specific adaptation, FORMED establishes a scalable and resource-efficient paradigm for foundation model repurposing in healthcare. This approach prioritizes clinical adaptability over rigid task-centric design, offering a practical pathway for real-world implementation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 34 of its 63 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

34 of 63 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  7. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  8. 2020Reformer: The Efficient Transformercited by 1,668
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  10. 2018Model Cards for Model Reportingcited by 1,080
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
22 more
  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  2. 2023A decoder-only foundation model for time-series forecastingcited by 692
  3. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  4. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  5. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  6. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  7. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  8. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  9. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  10. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  11. 2020MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classificationcited by 190
  12. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  13. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  14. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 168
  15. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  16. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 125
  17. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  18. 2023Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-Seriescited by 59
  19. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  20. 2023Multi-resolution Time-Series Transformer for Long-term Forecastingcited by 41
  21. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  22. 2023Large Transformers are Better EEG Learnerscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.