mlsanityData to 7 Oct 2026

Atla Selene Mini: A General Purpose Evaluation Model

arXiv 2501.17195 ↗First posted 27 Jan 2025cs.CL cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

We introduce Atla Selene Mini, a state-of-the-art small language model-as-a-judge (SLMJ). Selene Mini is a general-purpose evaluator that outperforms the best SLMJs and GPT-4o-mini on overall performance across 11 out-of-distribution benchmarks, spanning absolute scoring, classification, and pairwise preference tasks. It is the highest-scoring 8B generative model on RewardBench, surpassing strong baselines like GPT-4o and specialized judges. To achieve this, we develop a principled data curation strategy that augments public datasets with synthetically generated critiques and ensures high quality through filtering and dataset ablations. We train our model on a combined direct preference optimization (DPO) and supervised fine-tuning (SFT) loss, and produce a highly promptable evaluator that excels in real-world scenarios. Selene Mini shows dramatically improved zero-shot agreement with human expert evaluations on financial and medical industry datasets. It is also robust to variations in prompt format. Preliminary results indicate that Selene Mini is the top-ranking evaluator in a live, community-driven Judge Arena. We release the model weights on HuggingFace (https://hf.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B) and Ollama to encourage widespread community adoption.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 11.

27 papers.

Sort:

By area:

  1. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  2. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  3. 2025Reward Reasoning Modelcited by 47
  4. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  5. 2025The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authorscited by 16
  6. 2025Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactionscited by 12
  7. 2025Sherlock: Reliable and Efficient Agentic Workflow Executioncited by 10
  8. 2025InfoQuest: Evaluating Multi-Turn Dialogue Agents for Open-Ended Conversations with Hidden Contextcited by 8
  9. 2026Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimizationcited by 8
  10. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  11. 2025Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Modelscited by 5
  12. 2025DISCO: Disentangled Communication Steering for Large Language Modelscited by 5
15 more
  1. 2025Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domainscited by 5
  2. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluationcited by 5
  3. 2025Reference-Free Rating of LLM Responses via Latent Informationcited by 4
  4. 2025XBreaking: Understanding how LLMs security alignment can be brokencited by 3
  5. 2026A Finite-Calibration Regime Map for LLM Judge Panelscited by 2
  6. 2026SecureBreak -- A dataset towards safe and secure modelscited by 1
  7. 2026Comparing Developer and LLM Biases in Code Evaluationcited by 1
  8. 2026Stopping and Routing LLM Judge Panelscited by 1
  9. 2025Model selection meets clinical semantics: Optimizing ICD-10-CM prediction via LLM-as-Judge evaluation, redundancy-aware sampling, and section-aware fine-tuningcited by 0
  10. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  11. 2026VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inferencecited by 0
  12. 2026Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregationcited by 0
  13. 2026Counsel: A Meta-Evaluation Dataset for Agentic Taskscited by 0
  14. 2026Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeedscited by 0
  15. 2026Small Language Models as Judges for Rubric-Based Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SLMJury: Can Small Language Models Judge as Well as Large Ones?
  2. 2024Direct Judgement Preference Optimization
  3. 2026Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
  4. 2026JudgeArena: A Unified Framework for Reproducible LLM-Judge Evaluation
  5. 2024Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation
  6. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judges
  7. 2025Are We on the Right Way to Assessing LLM-as-a-Judge?
  8. 2026The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation