mlsanityData to 7 Oct 2026

Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews

arXiv 2104.05861 ↗First posted 12 Apr 2021cs.SE cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Context: Mobile app reviews written by users on app stores or social media are significant resources for app developers.Analyzing app reviews have proved to be useful for many areas of software engineering (e.g., requirement engineering, testing). Automatic classification of app reviews requires extensive efforts to manually curate a labeled dataset. When the classification purpose changes (e.g. identifying bugs versus usability issues or sentiment), new datasets should be labeled, which prevents the extensibility of the developed models for new desired classes/tasks in practice. Recent pre-trained neural language models (PTM) are trained on large corpora in an unsupervised manner and have found success in solving similar Natural Language Processing problems. However, the applicability of PTMs is not explored for app review classification Objective: We investigate the benefits of PTMs for app review classification compared to the existing models, as well as the transferability of PTMs in multiple settings. Method: We empirically study the accuracy and time efficiency of PTMs compared to prior approaches using six datasets from literature. In addition, we investigate the performance of the PTMs trained on app reviews (i.e. domain-specific PTMs) . We set up different studies to evaluate PTMs in multiple settings: binary vs. multi-class classification, zero-shot classification (when new labels are introduced to the model), multi-task setting, and classification of reviews from different resources. The datasets are manually labeled app review datasets from Google Play Store, Apple App Store, and Twitter data. In all cases, Micro and Macro Precision, Recall, and F1-scores will be used and we will report the time required for training and prediction with the models.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 59 of its 176 references that are in the map. First posted on arXiv 12 Apr 2021.

Builds on

59 of 176 references.

References · 59

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2018Deep contextualized word representationscited by 2,285
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  11. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  12. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
38 more
  1. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  2. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  3. 2018CoQA: A Conversational Question Answering Challengecited by 674
  4. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  5. 2019Text Summarization with Pretrained Encoderscited by 553
  6. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  7. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  8. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  9. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  10. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  11. 2017Learning to Represent Programs with Graphscited by 219
  12. 2020Extractive Summarization as Text Matchingcited by 189
  13. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  14. 2019Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentencecited by 121
  15. 2019HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarizationcited by 120
  16. 2020LEGAL-BERT: The Muppets straight out of Law Schoolcited by 111
  17. 2020Adversarial Training for Large Neural Language Modelscited by 109
  18. 2017Semi-supervised sequence tagging with bidirectional language modelscited by 86
  19. 2020Retrospective Reader for Machine Reading Comprehensioncited by 86
  20. 2020Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERTcited by 69
  21. 2019DocBERT: BERT for Document Classificationcited by 68
  22. 2017Learning to select data for transfer learning with Bayesian Optimizationcited by 64
  23. 2019Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classificationcited by 62
  24. 2019Classification and Clustering of Arguments with Contextualized Word Embeddingscited by 57
  25. 2019Exploiting BERT for End-to-End Aspect-based Sentiment Analysiscited by 57
  26. 2019Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documentscited by 57
  27. 2019"Mask and Infill" : Applying Masked Language Model to Sentiment Transfercited by 48
  28. 2019Pre-trained Language Model Representations for Language Generationcited by 34
  29. 2020Adversarial Training for Aspect-Based Sentiment Analysis with BERTcited by 26
  30. 2019On the use of BERT for Neural Machine Translationcited by 23
  31. 2018Categorizing the Content of GitHub README Filescited by 18
  32. 2018A Benchmark Study on Sentiment Analysis for Software Engineering Researchcited by 14
  33. 2020Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inferencecited by 14
  34. 2019Classifying Multilingual User Feedback using Traditional Machine Learning and Deep Learningcited by 10
  35. 2016Using Word Embeddings in Twitter Election Classificationcited by 8
  36. 2021Automatically Matching Bug Reports With Related App Reviewscited by 8
  37. 2017Auto-completing Bug Reports for Android Applicationscited by 7
  38. 2019An Investigation of Transfer Learning-Based Sentiment Analysis in Japanesecited by 7

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 1; 2024: 0; 2025: 2; 2026: 2.

7 papers.

  1. 2022On the Effectiveness of Pretrained Models for API Learningcited by 9
  2. 2025RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practicecited by 6
  3. 2022Pre-Trained Neural Language Models for Automatic Mobile App User Feedback Answer Generationcited by 2
  4. 2023APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentationcited by 2
  5. 2025Exploring Zero-Shot App Review Classification with ChatGPT: Challenges and Potentialcited by 1
  6. 2026Age Matters: Analyzing Age-Related Discussions in App Reviewscited by 0
  7. 2026From Reviews to Requirements: Can LLMs Generate Human-Like User Stories?cited by 0