mlsanityData to 7 Oct 2026

Efficient Multi-View Fusion and Flexible Adaptation to View Missing in Cardiovascular System Signals

arXiv 2406.08930 ↗First posted 13 Jun 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The progression of deep learning and the widespread adoption of sensors have facilitated automatic multi-view fusion (MVF) about the cardiovascular system (CVS) signals. However, prevalent MVF model architecture often amalgamates CVS signals from the same temporal step but different views into a unified representation, disregarding the asynchronous nature of cardiovascular events and the inherent heterogeneity across views, leading to catastrophic view confusion. Efficient training strategies specifically tailored for MVF models to attain comprehensive representations need simultaneous consideration. Crucially, real-world data frequently arrives with incomplete views, an aspect rarely noticed by researchers. Thus, the View-Centric Transformer (VCT) and Multitask Masked Autoencoder (M2AE) are specifically designed to emphasize the centrality of each view and harness unlabeled data to achieve superior fused representations. Additionally, we systematically define the missing-view problem for the first time and introduce prompt techniques to aid pretrained MVF models in flexibly adapting to various missing-view scenarios. Rigorous experiments involving atrial fibrillation detection, blood pressure estimation, and sleep staging-typical health monitoring tasks-demonstrate the remarkable advantage of our method in MVF compared to prevailing methodologies. Notably, the prompt technique requires finetuning less than 3% of the entire model's data, substantially fortifying the model's resilience to view missing while circumventing the need for complete retraining. The results demonstrate the effectiveness of our approaches, highlighting their potential for practical applications in cardiovascular health monitoring. Codes and models are released at URL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 60 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

31 of 60 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  12. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
19 more
  1. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  2. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  3. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
  4. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  5. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  6. 2021A Survey of Transformerscited by 399
  7. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  8. 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 222
  9. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  10. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  11. 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 170
  12. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  13. 2023To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisiscited by 88
  14. 2021Self-supervised representation learning from 12-lead ECG datacited by 80
  15. 2023TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoderscited by 42
  16. 2021Intra-Inter Subject Self-supervised Learning for Multivariate Cardiac Signalscited by 35
  17. 2023Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicinecited by 24
  18. 2021Evaluating Contrastive Learning on Wearable Timeseries for Downstream Clinical Outcomescited by 8
  19. 2023Contrastive Self-Supervised Learning Based Approach for Patient Similarity: A Case Study on Atrial Fibrillation Detection from PPG Signalcited by 1

Built on by

Not cited yet.