mlsanityData to 7 Oct 2026

Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024

arXiv 2408.05388 ↗First posted 9 Aug 2024cs.IRcited by 8

Not on the map. Why?

Abstract (arXiv)

The first edition of the workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) took place in July 2024, co-located with the ACM SIGIR Conference 2024 in the USA (SIGIR 2024). The aim was to bring information retrieval researchers together around the topic of LLMs for evaluation in information retrieval that gathered attention with the advancement of large language models and generative AI. Given the novelty of the topic, the workshop was focused around multi-sided discussions, namely panels and poster sessions of the accepted proceedings papers.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Aug 2024.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 6.

8 papers.

  1. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  2. 2025A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgmentcited by 15
  3. 2025Do RAG Systems Really Suffer From Positional Bias?cited by 12
  4. 2025Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggetscited by 8
  5. 2025Towards Understanding Bias in Synthetic Data for Evaluationcited by 7
  6. 2025Redefining Retrieval Evaluation in the Era of LLMscited by 3
  7. 2024Evaluating Cost-Accuracy Trade-offs in Multimodal Search Relevance Judgementscited by 1
  8. 2024Semantic Search Evaluationcited by 0