# RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems

arXiv: https://arxiv.org/abs/2601.19637 · HTML: https://mlsanity.com/paper/2601.19637/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings](https://mlsanity.com/paper/2202.06671/) (2022)
- [Matching Papers and Reviewers at Large Conferences](https://mlsanity.com/paper/2202.12273/) (2022)
- [More Robust Dense Retrieval with Contrastive Dual Learning](https://mlsanity.com/paper/2107.07773/) (2021)
- [A Gold Standard Dataset for the Reviewer Assignment Problem](https://mlsanity.com/paper/2303.16750/) (2023)
- [Chain-of-Factors Paper-Reviewer Matching](https://mlsanity.com/paper/2310.14483/) (2023)
- [A SUPER* Algorithm to Optimize Paper Bidding in Peer Review](https://mlsanity.com/paper/2007.07079/) (2020)
- [RevGNN: Negative Sampling Enhanced Contrastive Graph Learning for Academic Reviewer Recommendation](https://mlsanity.com/paper/2407.20684/) (2024)
- [Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions](https://mlsanity.com/paper/2412.06606/) (2024)
- [Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge](https://mlsanity.com/paper/2506.22644/) (2025)

## Built on by (most cited first, 2 of 2)

- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [MERIT: Matching Expertise via Rubric-Informed Training for Reviewer Assignment](https://mlsanity.com/paper/2605.27865/) (2026)

## Abstract (arXiv)

Reviewer assignment is increasingly critical yet challenging in the LLM era, where rapid topic shifts render many pre-2023 benchmarks outdated and where proxy signals poorly reflect true reviewer familiarity. We address this evaluation bottleneck by introducing LR-bench, a high-fidelity, up-to-date benchmark curated from 2024-2025 AI/NLP manuscripts with five-level self-assessed familiarity ratings collected via a large-scale email survey, yielding 1055 expert-annotated paper-reviewer-score annotations. We further propose RATE, a reviewer-centric ranking framework that distills each reviewer's recent publications into compact keyword-based profiles and fine-tunes an embedding model with weak preference supervision constructed from heuristic retrieval signals, enabling matching each manuscript against a reviewer profile directly. Across LR-bench and the CMU gold-standard dataset, our approach consistently achieves state-of-the-art performance, outperforming strong embedding baselines by a clear margin. We release LR-bench at https://huggingface.co/datasets/Gnociew/LR-bench, and a GitHub repository at https://github.com/Gnociew/RATE-Reviewer-Assign.