# Leveraging Convolutional Neural Network-Transformer Synergy for Predictive Modeling in Risk-Based Applications

arXiv: https://arxiv.org/abs/2412.18222 · HTML: https://mlsanity.com/paper/2412.18222/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 24 Dec 2024.

## Builds on (most cited first, 7 of 7)

- [Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining](https://mlsanity.com/paper/2409.14327/) (2024)
- [Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networks](https://mlsanity.com/paper/2410.17617/) (2024)
- [Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation](https://mlsanity.com/paper/2409.11703/) (2024)
- [LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuning](https://mlsanity.com/paper/2411.09947/) (2024)
- [Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparison](https://mlsanity.com/paper/2412.05536/) (2024)
- [Stock Type Prediction Model Based on Hierarchical Graph Neural Network](https://mlsanity.com/paper/2412.06862/) (2024)
- [Integrative Analysis of Financial Market Sentiment Using CNN and GRU for Risk Prediction and Alert Systems](https://mlsanity.com/paper/2412.10199/) (2024)

## Built on by (most cited first, 9 of 9)

- [Graph Neural Network-Driven Hierarchical Mining for Complex Imbalanced Data](https://mlsanity.com/paper/2502.03803/) (2025)
- [Social Network User Profiling for Anomaly Detection Based on Graph Neural Networks](https://mlsanity.com/paper/2503.19380/) (2025)
- [Detecting Credit Card Fraud via Heterogeneous Graph Neural Networks with Graph Attention](https://mlsanity.com/paper/2504.08183/) (2025)
- [Context-Aware Rule Mining Using a Dynamic Transformer-Based Framework](https://mlsanity.com/paper/2503.11125/) (2025)
- [Contrastive and Variational Approaches in Self-Supervised Learning for Complex Data Mining](https://mlsanity.com/paper/2504.04032/) (2025)
- [Multi-Level Attention and Contrastive Learning for Enhanced Text Classification with an Optimized Transformer](https://mlsanity.com/paper/2501.13467/) (2025)
- [A Deep Learning Approach to Anomaly Detection in High-Frequency Trading Data](https://mlsanity.com/paper/2504.00287/) (2025)
- [Multi-Level Service Performance Forecasting via Spatiotemporal Graph Neural Networks](https://mlsanity.com/paper/2508.07122/) (2025)
- [TRUST-SCF: Transformer-based Risk Understanding and Scoring for Transactional Supply Chain Finance](https://mlsanity.com/paper/2606.08140/) (2026)

## Abstract (arXiv)

With the development of the financial industry, credit default prediction, as an important task in financial risk management, has received increasing attention. Traditional credit default prediction methods mostly rely on machine learning models, such as decision trees and random forests, but these methods have certain limitations in processing complex data and capturing potential risk patterns. To this end, this paper proposes a deep learning model based on the combination of convolutional neural networks (CNN) and Transformer for credit user default prediction. The model combines the advantages of CNN in local feature extraction with the ability of Transformer in global dependency modeling, effectively improving the accuracy and robustness of credit default prediction. Through experiments on public credit default datasets, the results show that the CNN+Transformer model outperforms traditional machine learning models, such as random forests and XGBoost, in multiple evaluation indicators such as accuracy, AUC, and KS value, demonstrating its powerful ability in complex financial data modeling. Further experimental analysis shows that appropriate optimizer selection and learning rate adjustment play a vital role in improving model performance. In addition, the ablation experiment of the model verifies the advantages of the combination of CNN and Transformer and proves the complementarity of the two in credit default prediction. This study provides a new idea for credit default prediction and provides strong support for risk assessment and intelligent decision-making in the financial field. Future research can further improve the prediction effect and generalization ability by introducing more unstructured data and improving the model architecture.