mlsanityData to 7 Oct 2026

Researching Alignment Research: Unsupervised Analysis

arXiv 2206.02841 ↗First posted 6 Jun 2022cs.CY cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

AI alignment research is the field of study dedicated to ensuring that artificial intelligence (AI) benefits humans. As machine intelligence gets more advanced, this research is becoming increasingly important. Researchers in the field share ideas across different media to speed up the exchange of information. However, this focus on speed means that the research landscape is opaque, making it difficult for young researchers to enter the field. In this project, we collected and analyzed existing AI alignment research. We found that the field is growing quickly, with several subfields emerging in parallel. We looked at the subfields and identified the prominent researchers, recurring topics, and different modes of communication in each. Furthermore, we found that a classifier trained on AI alignment research articles can detect relevant articles that we did not originally include in the dataset. We are sharing the dataset with the research community and hope to develop tools in the future that will help both established researchers and young researchers get more involved in the field.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 16 of its 57 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

16 of 57 references.

References · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2021Ethical and social risks of harm from Language Modelscited by 957
  9. 2020Artificial Intelligence, Values and Alignmentcited by 443
  10. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  11. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  12. 2021Alignment of Language Agentscited by 167
4 more
  1. 2020Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimscited by 163
  2. 2022Compute Trends Across Three Eras of Machine Learningcited by 161
  3. 2017When Will AI Exceed Human Performance? Evidence from AI Expertscited by 41
  4. 2013The elephant in the room: multi-authorship and the assessment of individual researcherscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 3.

7 papers.

  1. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  2. 2023Getting aligned on representational alignmentcited by 142
  3. 2023Alignment with human representations supports robust few-shot learningcited by 25
  4. 2025Distributive Fairness in Large Language Models: Evaluating Alignment with Human Valuescited by 9
  5. 2025Position: Towards a Responsible LLM-empowered Multi-Agent Systemscited by 7
  6. 2024Improved Emotional Alignment of AI and Humans: Human Ratings of Emotions Expressed by Stable Diffusion v1, DALL-E 2, and DALL-E 3cited by 3
  7. 2025AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?cited by 1