mlsanityData to 14 Oct 2026

CoRAST: Towards Foundation Model-Powered Correlated Data Analysis in Resource-Constrained CPS and IoT

arXiv 2403.18451 ↗First posted 27 Mar 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Foundation models (FMs) emerge as a promising solution to harness distributed and diverse environmental data by leveraging prior knowledge to understand the complicated temporal and spatial correlations within heterogeneous datasets. Unlike distributed learning frameworks such as federated learning, which often struggle with multimodal data, FMs can transform diverse inputs into embeddings. This process facilitates the integration of information from various modalities and the application of prior learning to new domains. However, deploying FMs in resource-constrained edge systems poses significant challenges. To this end, we introduce CoRAST, a novel learning framework that utilizes FMs for enhanced analysis of distributed, correlated heterogeneous data. Utilizing a server-based FM, CoRAST can exploit existing environment information to extract temporal, spatial, and cross-modal correlations among sensor data. This enables CoRAST to offer context-aware insights for localized client tasks through FM-powered global representation learning. Our evaluation on real-world weather dataset demonstrates CoRAST's ability to exploit correlated heterogeneous data through environmental representation learning to reduce the forecast errors by up to 50.3% compared to the baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 27 Mar 2024.

Builds on

22 of 27 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2017Graph Attention Networkscited by 8,923
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  8. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  9. 2023A Comprehensive Overview of Large Language Modelscited by 648
  10. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  11. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  12. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
10 more
  1. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  2. 2023Multimodal Large Language Models: A Surveycited by 195
  3. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 172
  4. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 167
  5. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 131
  6. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  7. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  8. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
  9. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  10. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64

Built on by

Not cited yet.