# Contrastive Pre-training for Imbalanced Corporate Credit Ratings

arXiv: https://arxiv.org/abs/2102.12580 · HTML: https://mlsanity.com/paper/2102.12580/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 16 of 16)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [Rethinking the Value of Labels for Improving Class-Imbalanced Learning](https://mlsanity.com/paper/2006.07529/) (2020)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Selfie: Self-supervised Pretraining for Image Embedding](https://mlsanity.com/paper/1906.02940/) (2019)
- [DeepTrax: Embedding Graphs of Financial Transactions](https://mlsanity.com/paper/1907.07225/) (2019)
- [Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2010.14395/) (2020)
- [Application of Deep Neural Networks to assess corporate Credit Rating](https://mlsanity.com/paper/2003.02334/) (2020)
- [Every Corporation Owns Its Structure: Corporate Credit Ratings via Graph Neural Networks](https://mlsanity.com/paper/2012.01933/) (2020)
- [Every Corporation Owns Its Image: Corporate Credit Ratings via Convolutional Neural Networks](https://mlsanity.com/paper/2012.03744/) (2020)

## Built on by (most cited first, 4 of 4)

- [Adversarial Semi-supervised Learning for Corporate Credit Ratings](https://mlsanity.com/paper/2104.02479/) (2021)
- [Corporate Credit Rating: A Survey](https://mlsanity.com/paper/2309.14349/) (2023)
- [Scale-Translation Equivariant Network for Oceanic Internal Solitary Wave Localization](https://mlsanity.com/paper/2406.13060/) (2024)
- [CreditARF: A Framework for Corporate Credit Rating with Annual Report and Financial Feature Integration](https://mlsanity.com/paper/2508.02738/) (2025)

## Abstract (arXiv)

Corporate credit rating reflects the level of corporate credit and plays a crucial role in modern financial risk control. But real-world credit rating data usually shows long-tail distributions, which means heavy class imbalanced problem challenging the corporate credit rating system greatly. To tackle that, inspried by the recent advances of pre-train techniques in self-supervised representation learning, we propose a novel framework named Contrastive Pre-training for Corporate Credit Rating (CP4CCR), which utilizes the self-surpervision for getting over class imbalance. Specifically, we propose to, in the first phase, exert constrastive self-superivised pre-training without label information, which want to learn a better class-agnostic initialization. During this phase, two self-supervised task are developed within CP4CCR: (i) Feature Masking (FM) and (ii) Feature Swapping(FS). In the second phase, we can train any standard corporate redit rating model initialized by the pre-trained network. Extensive experiments conducted on the Chinese public-listed corporate rating dataset, prove that CP4CCR can improve the performance of standard corporate credit rating models, especially for class with few samples.