mlsanityData to 7 Oct 2026

Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research

arXiv 2604.17843 ↗First posted 20 Apr 2026cs.HC cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA's multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4-3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized "evidence engine"; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for "ecosystem-aware" Humble AI.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 62 of its 123 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

62 of 123 references.

References · 62

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  8. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  9. 2017Selective Classification for Deep Neural Networkscited by 799
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  11. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  12. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
38 more
  1. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  2. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  3. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  4. 2022Teaching language models to support answers with verified quotescited by 290
  5. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  6. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  7. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  8. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  9. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  10. 2024What Evidence Do Language Models Find Convincing?cited by 74
  11. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  12. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 56
  13. 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 55
  14. 2019Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processescited by 53
  15. 2025Toward an Evaluation Science for Generative AI Systemscited by 42
  16. 2023Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preferencecited by 37
  17. 2023Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledgecited by 32
  18. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  19. 2024Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanationscited by 30
  20. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  21. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  22. 2023The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive Visualizationcited by 23
  23. 2023RELIC: Investigating Large Language Model Responses using Self-Consistencycited by 23
  24. 2024Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generationcited by 21
  25. 2024Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challengescited by 20
  26. 2021Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisionscited by 18
  27. 2022Neural Approaches to Multilingual Information Retrievalcited by 18
  28. 2024PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Paperscited by 18
  29. 2024Task Supportive and Personalized Human-Large Language Model Interaction: A User Studycited by 17
  30. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  31. 2024Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generationscited by 14
  32. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  33. 2024Learning Fine-Grained Grounded Citations for Attributed Large Language Modelscited by 12
  34. 2024Designing for Appropriate Reliance: The Roles of AI Uncertainty Presentation, Initial User Decision, and User Demographics in AI-Assisted Decision-Makingcited by 11
  35. 2024HILL: A Hallucination Identifier for Large Language Modelscited by 11
  36. 2025Question Decomposition for Retrieval-Augmented Generationcited by 11
  37. 2025Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigationcited by 8
  38. 2025Beneficial Reasoning Behaviors in Agentic Search and Effective Post-training to Obtain Themcited by 7

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflowscited by 0