# Detecting Generated Scientific Papers using an Ensemble of Transformer Models

arXiv: https://arxiv.org/abs/2209.08283 · HTML: https://mlsanity.com/paper/2209.08283/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian](https://mlsanity.com/paper/2206.01583/) (2022)
- [Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals](https://mlsanity.com/paper/2107.06751/) (2021)
- [g2tmn at Constraint@AAAI2021: Exploiting CT-BERT and Ensembling Learning for COVID-19 Fake News Detection](https://mlsanity.com/paper/2012.11967/) (2020)
- [Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation](https://mlsanity.com/paper/2201.05302/) (2022)
- [X-SCITLDR: Cross-Lingual Extreme Summarization of Scholarly Documents](https://mlsanity.com/paper/2205.15051/) (2022)
- [BERT-based Acronym Disambiguation with Multiple Training Strategies](https://mlsanity.com/paper/2103.00488/) (2021)
- [AT-BERT: Adversarial Training BERT for Acronym Identification Winning Solution for SDU@AAAI-21](https://mlsanity.com/paper/2101.03700/) (2021)
- [Applying Transformer-based Text Summarization for Keyphrase Generation](https://mlsanity.com/paper/2209.03791/) (2022)

## Built on by (most cited first, 1 of 1)

- [An Ensemble Method Based on the Combination of Transformers with Convolutional Neural Networks to Detect Artificially Generated Text](https://mlsanity.com/paper/2310.17312/) (2023)

## Abstract (arXiv)

The paper describes neural models developed for the DAGPap22 shared task hosted at the Third Workshop on Scholarly Document Processing. This shared task targets the automatic detection of generated scientific papers. Our work focuses on comparing different transformer-based models as well as using additional datasets and techniques to deal with imbalanced classes. As a final submission, we utilized an ensemble of SciBERT, RoBERTa, and DeBERTa fine-tuned using random oversampling technique. Our model achieved 99.24% in terms of F1-score. The official evaluation results have put our system at the third place.