mlsanityData to 7 Oct 2026

Supervising strong learners by amplifying weak experts

arXiv 1810.08575 ↗First posted 19 Oct 2018cs.LG cs.AI stat.MLcited by 139

Not on the map. Why?

Cited by 46 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior. One solution is to have humans provide a training signal by demonstrating or judging performance, but this approach fails if the task is too complicated for a human to directly evaluate. We propose Iterated Amplification, an alternative training strategy which progressively builds up a training signal for difficult problems by combining solutions to easier subproblems. Iterated Amplification is closely related to Expert Iteration (Anthony et al., 2017; Silver et al., 2017), except that it uses no external reward function. We present results in algorithmic environments, showing that Iterated Amplification can efficiently learn complex behaviors.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 1; 2021: 7; 2022: 15; 2023: 16; 2024: 25; 2025: 41; 2026: 32.

139 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  9. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  11. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  12. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
38 more
  1. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  2. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  4. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  5. 2022Self-critiquing models for assisting human evaluatorscited by 372
  6. 2022Teaching language models to support answers with verified quotescited by 290
  7. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  8. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  9. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  10. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  11. 2023Large Language Model Alignment: A Surveycited by 241
  12. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  13. 2022TALM: Tool Augmented Language Modelscited by 197
  14. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  15. 2021Alignment of Language Agentscited by 167
  16. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 120
  17. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 111
  18. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  19. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  20. 2022Is Power-Seeking AI an Existential Risk?cited by 101
  21. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  22. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  23. 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
  24. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 73
  25. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  26. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  27. 2023Scalable AI Safety via Doubly-Efficient Debatecited by 54
  28. 2024WARP: On the Benefits of Weight Averaged Rewarded Policiescited by 42
  29. 2025Teaching Language Models to Critique via Reinforcement Learningcited by 38
  30. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  31. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  32. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibrationcited by 30
  33. 2024Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Expertscited by 28
  34. 2023Compositional preference models for aligning LMscited by 25
  35. 2024Detecting Mode Collapse in Language Models via Narrationcited by 24
  36. 2024A Survey on Human Preference Learning for Large Language Modelscited by 23
  37. 2023Assessing Large Language Models on Climate Informationcited by 22
  38. 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 21

50 most cited of 139 All 139 (JSON)