mlsanityData to 14 Oct 2026

Pytrec_eval: An Extremely Fast Python Interface to trec_eval

arXiv 1805.01597 ↗First posted 4 May 2018cs.IRcited by 46

Not on the map. Why?

Abstract (arXiv)

We introduce pytrec_eval, a Python interface to the tree_eval information retrieval evaluation toolkit. pytrec_eval exposes the reference implementations of trec_eval within Python as a native extension. We show that pytrec_eval is around one order of magnitude faster than invoking trec_eval as a sub process from within Python. Compared to a native Python implementation of NDCG, pytrec_eval is twice as fast for practically-sized rankings. Finally, we demonstrate its effectiveness in an application where pytrec_eval is combined with Pyndri and the OpenAI Gym where query expansion is learned using Q-learning.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). First posted on arXiv 4 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 7; 2024: 16; 2025: 11; 2026: 6.

46 papers.

Sort:

By area:

  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
  3. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  4. 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
  5. 2024CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Searchcited by 29
  6. 2023Learning to Relate to Previous Turns in Conversational Searchcited by 26
  7. 2024Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversationcited by 23
  8. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 22
  9. 2024Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answerscited by 15
  10. 2024AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignmentcited by 14
  11. 2024How to Leverage Personal Textual Knowledge for Personalized Conversational Information Retrievalcited by 11
  12. 2024Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Searchcited by 11
34 more
  1. 2023DAPR: A Benchmark on Document-Aware Passage Retrievalcited by 9
  2. 2024ConvSDG: Session Data Generation for Conversational Searchcited by 8
  3. 2024A Reproducibility Study of PLAIDcited by 8
  4. 2025mFollowIR: a Multilingual Benchmark for Instruction Following in Retrievalcited by 7
  5. 2022From Easy to Hard: A Dual Curriculum Learning Framework for Context-Aware Document Rankingcited by 5
  6. 2025Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Modelscited by 5
  7. 2025ConvMix: A Mixed-Criteria Data Augmentation Framework for Conversational Dense Retrievalcited by 5
  8. 2022Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Rankingcited by 4
  9. 2024History-Aware Conversational Dense Retrievalcited by 4
  10. 2024Learning Job Title Representation from Job Description Aggregation Networkcited by 4
  11. 2024DEXTER: A Benchmark for open-domain Complex Question Answering using LLMscited by 4
  12. 2024Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Modelcited by 4
  13. 2023LePaRD: A Large-Scale Dataset of Judges Citing Precedentscited by 3
  14. 2025Unify Graph Learning with Text: Unleashing LLM Potentials for Session Searchcited by 3
  15. 2025SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questionscited by 3
  16. 2024Conversational Query Reformulation with the Guidance of Retrieved Documentscited by 2
  17. 2025AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptationcited by 2
  18. 2024URL: Universal Referential Knowledge Linking via Task-instructed Representation Compressioncited by 1
  19. 2024Numbers Matter! Bringing Quantity-awareness to Retrieval Systemscited by 1
  20. 2025Adaptive Personalized Conversational Information Retrievalcited by 1
  21. 2026Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrievalcited by 1
  22. 2021Introducing Information Retrieval for Biomedical Informatics Studentscited by 0
  23. 2022Deep Reinforcement Agent for Efficient Instant Searchcited by 0
  24. 2023Empowering Language Model with Guided Knowledge Fusion for Biomedical Document Re-rankingcited by 0
  25. 2023How Can Context Help? Exploring Joint Retrieval of Passage and Personalized Contextcited by 0
  26. 2024Query-oriented Data Augmentation for Session Searchcited by 0
  27. 2025References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulationcited by 0
  28. 2025Bridge the Gap between Past and Future: Siamese Model Optimization for Context-Aware Document Rankingcited by 0
  29. 2025CLAP: Coreference-Linked Augmentation for Passage Retrievalcited by 0
  30. 2026Caraman at SemEval-2026 Task 8: Three-Stage Multi-Turn Retrieval with Query Rewriting, Hybrid Search, and Cross-Encoder Rerankingcited by 0
  31. 2026RCEM: Robust Conversational Search EMbedder in Distributional Shiftcited by 0
  32. 2026Improving Ad-hoc Search Effectiveness for Conversational Information Retrieval via Model Mergingcited by 0
  33. 2026When Do Anchor-Based Pointwise LLM Rerankers Help? Retriever Quality, Statistical Scope, and Anchor Designcited by 0
  34. 2026CompCQR: Compositional Query Generation for Training-Free Conversational Searchcited by 0