mlsanityData to 14 Oct 2026

Learning Transactions Representations for Information Management in Banks: Mastering Local, Global, and External Knowledge

arXiv 2404.02047 ↗First posted 2 Apr 2024cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

In today's world, banks use artificial intelligence to optimize diverse business processes, aiming to improve customer experience. Most of the customer-related tasks can be categorized into two groups: 1) local ones, which focus on a client's current state, such as transaction forecasting, and 2) global ones, which consider the general customer behaviour, e.g., predicting successful loan repayment. Unfortunately, maintaining separate models for each task is costly. Therefore, to better facilitate information management, we compared eight state-of-the-art unsupervised methods on 11 tasks in search for a one-size-fits-all solution. Contrastive self-supervised learning methods were demonstrated to excel at global problems, while generative techniques were superior at local tasks. We also introduced a novel approach, which enriches the client's representation by incorporating external information gathered from other clients. Our method outperforms classical models, boosting accuracy by up to 20%.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 34 of its 74 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

34 of 74 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  9. 2016Pixel Recurrent Neural Networkscited by 816
  10. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  11. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  12. 2020Self-supervised Learning: Generative or Contrastivecited by 694
22 more
  1. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  2. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  3. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  4. 2020Transformer Hawkes Processcited by 235
  5. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  6. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  7. 2018Recent Advances in Autoencoder-Based Representation Learningcited by 133
  8. 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 93
  9. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  10. 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 68
  11. 2020An Evaluation of Change Point Detection Algorithmscited by 63
  12. 2020Time Series Change Point Detection with Self-Supervised Contrastive Predictive Codingcited by 58
  13. 2020Learning from Irregularly-Sampled Time Series: A Missing Data Perspectivecited by 40
  14. 2020CoLES: Contrastive Learning for Event Sequences with Self-Supervisioncited by 32
  15. 2019E.T.-RNN: Applying Deep Learning to Credit Loan Applicationscited by 26
  16. 2022Similarity learning for wells based on logging datacited by 6
  17. 2022Deep Learning vs. Gradient Boosting: Benchmarking state-of-the-art machine learning algorithms for credit scoringcited by 6
  18. 2019Learning Latent Representations of Bank Customers With The Variational Autoencodercited by 4
  19. 2021InDiD: Instant Disorder Detection via Representation Learningcited by 4
  20. 2019A comparison of Deep Learning performances with other machine learning algorithms on credit scoring unbalanced datacited by 3
  21. 2021Bank transactions embeddings help to uncover current macroeconomicscited by 3
  22. 2024Meta-Learning for Neural Network-based Temporal Point Processescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 3.

6 papers.

  1. 2024Multimodal Banking Dataset: Understanding Client Needs through Event Sequencescited by 9
  2. 2024Uniting contrastive and generative learning for event sequences modelscited by 2
  3. 2025Looking around you: external information enhances representations for event sequencescited by 0
  4. 2026Beyond Isolated Clients: Integrating Graph-Based Embeddings into Event Sequence Modelscited by 0
  5. 2026A Foundation Model for Multimodal Event Sequences in Financial Applicationscited by 0
  6. 2026Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspectivecited by 0