mlsanityData to 14 Oct 2026

RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation

arXiv 2601.04350 ↗First posted 7 Jan 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Scientific rigour tends to be sidelined in favour of bold statements, leading authors to overstate claims beyond what their results support. We present RIGOURATE, a two-stage multimodal framework that retrieves supporting evidence from a paper's body and assigns each claim an overstatement score. The framework consists of a dataset of over 10K claim-evidence sets from ICLR and NeurIPS papers, annotated using eight LLMs, with overstatement scores calibrated using peer-review comments and validated through human evaluation. It employes a fine-tuned reranker for evidence retrieval and a fine-tuned model to predict overstatement scores with justification. Compared to strong baselines, RIGOURATE enables improved evidence retrieval and overstatement detection. Overall, our work operationalises evidential proportionality and supports clearer, more transparent scientific communication.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 36 of its 87 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

36 of 87 references.

References · 36

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  3. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  6. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  7. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  8. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  9. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  10. 2024Generative Representational Instruction Tuningcited by 233
  11. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
  12. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
24 more
  1. 2019Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approachescited by 185
  2. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
  3. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  4. 2023LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scorescited by 90
  5. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  6. 2020ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesiscited by 54
  7. 2023Scientific Fact-Checking: A Survey of Resources and Approachescited by 54
  8. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  9. 2023Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experimentcited by 34
  10. 2021Cross-replication Reliability -- An Empirical Approach to Interpreting Inter-rater Reliabilitycited by 31
  11. 2024The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representationcited by 29
  12. 2021Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communicationscited by 27
  13. 2023Summarizing Multiple Documents with Conversational Structure for Meta-Review Generationcited by 27
  14. 2023Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experimentscited by 27
  15. 2024AAAR-1.0: Assessing AI's Potential to Assist Researchcited by 24
  16. 2023The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Whocited by 19
  17. 2022On Isotropy, Contextualization and Learning Dynamics of Contrastive-based Sentence Representation Learningcited by 15
  18. 2025Evaluating Large Language Models as Expert Annotatorscited by 9
  19. 2023On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Researchcited by 7
  20. 2024How We Refute Claims: Automatic Fact-Checking through Flaw Identification and Explanationcited by 7
  21. 2025A Case Study of Scalable Content Annotation Using Multi-LLM Consensus and Human Reviewcited by 7
  22. 2024On the Rigour of Scientific Writing: Criteria, Analysis, and Insightscited by 5
  23. 2025E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Searchcited by 4
  24. 2025FRONTIER-RevRec: A Large-scale Dataset for Reviewer Recommendationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5
  2. 2026A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluationcited by 4
  3. 2026When Evidence Changes the Subject: Subject-Typed Claim Licensing for Learned Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Missci: Reconstructing Fallacies in Misrepresented Science
  2. 2024Grounding Fallacies Misrepresenting Scientific Publications in Evidence
  3. 2025Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim rightarrow Evidence Reasoning
  4. 2026SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scores
  5. 2026When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification
  6. 2026DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation
  7. 2025Format Matters: The Robustness of Multimodal LLMs in Reviewing Evidence from Tables and Charts
  8. 2025MuSciClaims: Multimodal Scientific Claim Verification