MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

Machine-Generated Text (MGT) is becoming increasingly difficult to distinguish from Human-Written Text (HWT). This trend has exacerbated malicious activities such as fake news and online fraud. The generalization ability of fine-tuned detectors relies heavily on dataset quality, and simply expanding the sources of MGT may become increasingly insufficient. Further augmentation of the generation process is required. Based on HC-Var's theory, enhancing the human-like alignment of MGT not only facilitates robustness testing of existing detectors but also boosts the generalization ability of detectors fine-tuned on such aligned MGT datasets. Therefore, we propose the Machine-Augment-Generated Text via Alignment (MAGA) Detection Benchmark. MAGA integrates several alignment methods, ranging from prompt construction to Generator-Detector Adversarial Reinforcement Learning (GDARL) and the reasoning process. In our experiments, the RoBERTa detector fine-tuned on MAGA achieves an average improvement of 4.60% in generalization AUC. Conversely, the aligned MGTs in MAGA also lead to an average decrease of 8.13% in the AUC of selected detectors. We hope the MAGA Benchmark will provide valuable insights for future research on the generalization ability of MGT detectors.