mlsanityData to 14 Oct 2026

MEANT: Multimodal Encoder for Antecedent Information

arXiv 2411.06616 ↗First posted 10 Nov 2024cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The stock market provides a rich well of information that can be split across modalities, making it an ideal candidate for multimodal evaluation. Multimodal data plays an increasingly important role in the development of machine learning and has shown to positively impact performance. But information can do more than exist across modes -- it can exist across time. How should we attend to temporal data that consists of multiple information types? This work introduces (i) the MEANT model, a Multimodal Encoder for Antecedent information and (ii) a new dataset called TempStock, which consists of price, Tweets, and graphical data with over a million Tweets from all of the companies in the S&P 500 Index. We find that MEANT improves performance on existing baselines by over 15%, and that the textual information affects performance far more than the visual information on our time-dependent task from our ablation study.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 64 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

28 of 64 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2015Fast R-CNNcited by 2,933
  8. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2017Decoupled Weight Decay Regularizationcited by 1,831
  11. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
16 more
  1. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  2. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  3. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  4. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 385
  5. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  6. 2019Understanding and Improving Layer Normalizationcited by 217
  7. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 194
  8. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  9. 2019Large Scale Structure of Neural Network Loss Landscapescited by 68
  10. 2018The Goldilocks zone: Towards better understanding of neural network loss landscapescited by 32
  11. 2022Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Informationcited by 31
  12. 2023Multimodal Multi-loss Fusion Network for Sentiment Analysiscited by 27
  13. 2022Layer or Representation Space: What makes BERT-based Evaluation Metrics Robust?cited by 10
  14. 2021TEASEL: A Transformer-Based Speech-Prefixed Language Modelcited by 8
  15. 2015Twitter Sentiment Analysis Applied to Finance: A Case Study in the Retail Industrycited by 3
  16. 2022Machine Learning for Stock Prediction Based on Fundamental Analysiscited by 3

Built on by

Not cited yet.