mlsanityData to 7 Oct 2026

Catalyst Papers in Artificial Intelligence Research: A Landscape on ICLR from 2017 to 2025

arXiv 2607.05401 ↗First posted 24 May 2026cs.DL cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

A small number of methodological contributions, including word2vec, the Transformer, large-scale pre-training, and reinforcement learning from human feedback, have reshaped NLP and AI research over the past decade. OpenReview now makes numeric reviewer scores and accept/reject decisions public for every ICLR submission. Whether such review signals identify trajectory-changing papers at submission time, however, remains untested at corpus scale. We answer this question on 36,113 papers from ICLR 2017--2025, identifying catalysts: papers whose descendants measurably redirect future research. We compare four disruptiveness measures (the Consolidation/Destabilization (CD) index, node2vec, the direction-aware Embedding Disruptiveness Measure (EDM), and an LLM-based semantic rater) and define a five-type operational catalyst taxonomy (topic initiator, topic bridge, within-topic redirector, simultaneous, and recognition-misaligned). EDM leads at identifying highly cited ICLR papers (AUC 0.83 vs. 0.60 for CD, 0.49 for node2vec, and 0.42 for the LLM rater). Topic initiators precede a 7.55× topic-share growth and topic bridges precede an 11.52× growth in cross-topic citation flow versus year-matched controls. We found that the peer review scores are essentially orthogonal to future disruptiveness (|ρ|≤0.005; accepted and rejected papers have indistinguishable mean EDM, p=0.11).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 13 of its 32 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

13 of 32 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  6. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  7. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 166
  8. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  9. 2021Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experimentcited by 49
  10. 2020An Open Review of OpenReview: A Critical Analysis of the Machine Learning Conference Review Processcited by 23
  11. 2015Defining and identifying Sleeping Beauties in sciencecited by 11
  12. 2023The disruption index is biased by citation inflationcited by 2
1 more
  1. 2025Uncovering simultaneous breakthroughs with a robust measure of disruptivenesscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Topical Phase Transitions in Artificial Intelligence Research: Large-Scale Evidence and an Early-Warning Signature for Emerging Topics
  2. 2026Do Reviewers Still Reward Lexical Complexity? A Frozen-Rater Study of Preference Drift in 124K ICLR Reviews
  3. 2026Paper Espresso: From Paper Overload to Research Insight
  4. 2025CS-PaperSum: A Large-Scale Dataset of AI-Generated Summaries for Scientific Papers
  5. 2026Benchmarking Agentic Review Systems
  6. 2026Does Multi-Agent Debate Improve AI Feedback on Research Papers?
  7. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing
  8. 2025ELAIPBench: A Benchmark for Expert-Level Artificial Intelligence Paper Understanding