mlsanityData to 7 Oct 2026

Performance Comparison of Simple Transformer and Res-CNN-BiLSTM for Cyberbullying Classification

arXiv 2206.02206 ↗First posted 5 Jun 2022cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The task of text classification using Bidirectional based LSTM architectures is computationally expensive and time consuming to train. For this, transformers were discovered which effectively give good performance as compared to the traditional deep learning architectures. In this paper we present a performance based comparison between simple transformer based network and Res-CNN-BiLSTM based network for cyberbullying text classification problem. The results obtained show that transformer we trained with 0.65 million parameters has significantly being able to beat the performance of Res-CNN-BiLSTM with 48.82 million parameters for faster training speeds and more generalized metrics. The paper also compares the 1-dimensional character level embedding network and 100-dimensional glove embedding network with transformer.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 60 references that are in the map. First posted on arXiv 5 Jun 2022.

Builds on

26 of 60 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  8. 2013Network In Networkcited by 600
  9. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 281
  10. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
  11. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  12. 2020k-Nearest Neighbour Classifiers: 2nd Edition (with Python examples)cited by 56
14 more
  1. 2020Multi-Head Attention: Collaborate Instead of Concatenatecited by 56
  2. 2019Word Embeddings: A Surveycited by 45
  3. 2020A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Modelscited by 30
  4. 2018Similarity encoding for learning with dirty categorical variablescited by 28
  5. 2020Neural Machine Translation: A Review of Methods, Resources, and Toolscited by 18
  6. 2015Max-Pooling Dropout for Regularization of Convolutional Neural Networkscited by 11
  7. 2021Recent Trends in Named Entity Recognition (NER)cited by 9
  8. 2022Succinct Differentiation of Disparate Boosting Ensemble Learning Methods for Prognostication of Polycystic Ovary Syndrome Diagnosiscited by 9
  9. 2022Discriminant Analysis in Contrasting Dimensions for Polycystic Ovary Syndrome Prognosticationcited by 9
  10. 2022Binary Classification for High Dimensional Data using Supervised Non-Parametric Ensemble Methodcited by 8
  11. 2018Selective Feature Connection Mechanism: Concatenating Multi-layer CNN Features with a Feature Selectorcited by 4
  12. 2022Combining Varied Learners for Binary Classification using Stacked Generalizationcited by 4
  13. 2022Effects of Parametric and Non-Parametric Methods on High Dimensional Sparse Matrix Representationscited by 2
  14. 2022Res-CNN-BiLSTM Network for overcoming Mental Health Disturbances caused due to Cyberbullying through Social Mediacited by 1

Built on by

Not cited yet.