# Real-World Deployment and Evaluation of Kwame for Science, An AI Teaching Assistant for Science Education in West Africa

arXiv: https://arxiv.org/abs/2302.10786 · HTML: https://mlsanity.com/paper/2302.10786/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection](https://mlsanity.com/paper/1911.04118/) (2019)
- [AI-Powered Learning: Making Education Accessible, Affordable, and Achievable](https://mlsanity.com/paper/2006.01908/) (2020)
- [Kwame: A Bilingual AI Teaching Assistant for Online SuaCode Courses](https://mlsanity.com/paper/2010.11387/) (2020)
- [SuaCode Africa: Teaching Coding Online to Africans using Smartphones](https://mlsanity.com/paper/2107.12257/) (2021)
- [Kwame for Science: An AI Teaching Assistant Based on Sentence-BERT for Science Education in West Africa](https://mlsanity.com/paper/2206.13703/) (2022)

## Built on by (most cited first, 5 of 5)

- [LittleMu: Deploying an Online Virtual Teaching Assistant via Heterogeneous Sources Integration and Chain of Teach Prompts](https://mlsanity.com/paper/2308.05935/) (2023)
- [Brilla AI: AI Contestant for the National Science and Maths Quiz](https://mlsanity.com/paper/2403.01699/) (2024)
- [Leveraging AI to Advance Science and Computing Education across Africa: Challenges, Progress and Opportunities](https://mlsanity.com/paper/2402.07397/) (2024)
- [GenAITEd Ghana: A First-of-Its-Kind Context-Aware and Curriculum-Aligned Conversational AI Agent for Teacher Education](https://mlsanity.com/paper/2601.06093/) (2025)
- [Adesua: Development and Feasibility Study of an AI WhatsApp Bot for Science Learning in West Africa](https://mlsanity.com/paper/2605.15376/) (2026)

## Abstract (arXiv)

Africa has a high student-to-teacher ratio which limits students' access to teachers for learning support such as educational question answering. In this work, we extended Kwame, a bilingual AI teaching assistant for coding education, adapted it for science education, and deployed it as a web app. Kwame for Science provides passages from well-curated knowledge sources and related past national exam questions as answers to questions from students based on the Integrated Science subject of the West African Senior Secondary Certificate Examination (WASSCE). Furthermore, students can view past national exam questions along with their answers and filter by year, question type, and topics that were automatically categorized by a topic detection model which we developed (91% unweighted average recall). We deployed Kwame for Science in the real world over 8 months and had 750 users across 32 countries (15 in Africa) and 1.5K questions asked. Our evaluation showed an 87.2% top 3 accuracy (n=109 questions) implying that Kwame for Science has a high chance of giving at least one useful answer among the 3 displayed. We categorized the reasons the model incorrectly answered questions to provide insights for future improvements. We also share challenges and lessons with the development, deployment, and human-computer interaction component of such a tool to enable other researchers to deploy similar tools. With a first-of-its-kind tool within the African context, Kwame for Science has the potential to enable the delivery of scalable, cost-effective, and quality remote education to millions of people across Africa.