mlsanityData to 14 Oct 2026

CRL+: A Novel Semi-Supervised Deep Active Contrastive Representation Learning-Based Text Classification Model for Insurance Data

arXiv 2302.04343 ↗First posted 8 Feb 2023cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Financial sector and especially the insurance industry collect vast volumes of text on a daily basis and through multiple channels (their agents, customer care centers, emails, social networks, and web in general). The information collected includes policies, expert and health reports, claims and complaints, results of surveys, and relevant social media posts. It is difficult to effectively extract label, classify, and interpret the essential information from such varied and unstructured material. Therefore, the Insurance Industry is among the ones that can benefit from applying technologies for the intelligent analysis of free text through Natural Language Processing (NLP). In this paper, CRL+, a novel text classification model combining Contrastive Representation Learning (CRL) and Active Learning is proposed to handle the challenge of using semi-supervised learning for text classification. In this method, supervised (CRL) is used to train a RoBERTa transformer model to encode the textual data into a contrastive representation space and then classify using a classification layer. This (CRL)-based transformer model is used as the base model in the proposed Active Learning mechanism to classify all the data in an iterative manner. The proposed model is evaluated using unstructured obituary data with objective to determine the cause of the death from the data. This model is compared with the CRL model and an Active Learning model with the RoBERTa base model. The experiment shows that the proposed method can outperform both methods for this specific task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 8 Feb 2023.

Builds on

31 of 48 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2018Deep contextualized word representationscited by 2,290
  12. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
19 more
  1. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  2. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  3. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  4. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 306
  5. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  6. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
  7. 2015Semi-Supervised Learning with Ladder Networkscited by 247
  8. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  9. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  10. 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 141
  11. 2015Character-Aware Neural Language Modelscited by 132
  12. 2019Semantics-aware BERT for Language Understandingcited by 112
  13. 2020Adversarial Training for Large Neural Language Modelscited by 111
  14. 2019TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selectioncited by 74
  15. 2018On the Dimensionality of Word Embeddingcited by 46
  16. 2016Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attentioncited by 23
  17. 2021SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representationscited by 7
  18. 2020A survey on natural language processing (nlp) and applications in insurancecited by 3
  19. 2016Modelling Interaction of Sentence Pair with coupled-LSTMscited by 1

Built on by

Not cited yet.