mlsanityData to 7 Oct 2026

Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction

arXiv 2604.07964 ↗First posted 9 Apr 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The integration of large language models (LLMs) into peer review raises a concern beyond authorship and detection: the potential cascading automation of the entire editorial process. As reviews become partially or fully machine-generated, it becomes plausible that editorial decisions may also be delegated to algorithmic systems, leading to a fully automated evaluation pipeline. They risk reshaping the criteria by which scientific work is assessed. This paper argues that machine-driven assessment may systematically favor standardized, pattern-conforming research while penalizing unconventional and paradigm-shifting ideas that require contextual human judgment. We consider that this shift could lead to epistemic homogenization, where researchers are implicitly incentivized to optimize their work for algorithmic approval rather than genuine discovery. To address this risk, we introduce an explainable framework (RAG-XAI) for assessing review quality and detecting automated patterns using markers LLM extractor, aiming to preserve transparency, accountability and creativity in science. The proposed framework achieves near-perfect detection performance, with XGBoost, Random Forest and LightGBM reaching 99.61% accuracy, AUC-ROC above 0.999 and F1-scores of 0.9925 on the test set, while maintaining extremely low false positive rates (<0.23%) and false negative rates (~0.8%). In contrast, the logistic regression baseline performs substantially worse (89.97% accuracy, F1-score 0.8314). Feature importance and SHAP analyses identify absence of personal signals and repetition patterns as the dominant predictors. Additionally, the RAG component achieves 90.5% top-1 retrieval accuracy, with strong same-class clustering in the embedding space, further supporting the reliability of the framework's outputs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 41 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

16 of 41 references.

References · 16

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  5. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  6. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 135
  7. 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 92
  8. 2024The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Ratescited by 66
  9. 2025Large language models for automated scholarly paper review: A surveycited by 59
  10. 2024Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Reviewcited by 19
  11. 2025Detecting LLM-Generated Peer Reviewscited by 18
  12. 2025Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attackscited by 14
4 more
  1. 2025Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviewscited by 10
  2. 2025Graph-Guided Passage Retrieval for Author-Centric Structured Feedbackcited by 3
  3. 2025Gen-Review: A Large-scale Dataset of AI-Generated (and Human-written) Peer Reviewscited by 3
  4. 2026Detecting AI-Generated Content in Academic Peer Reviewscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PeerPrism: Peer Evaluation Expertise vs Review-writing AI
  2. 2025AI and the Future of Academic Peer Review
  3. 2026LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challenges
  4. 2025Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework
  5. 2025Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback
  6. 2026From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent
  7. 2026ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review
  8. 2026PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers