# Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needs

arXiv: https://arxiv.org/abs/2411.19554 · HTML: https://mlsanity.com/paper/2411.19554/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [Evaluation of Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2405.07437/) (2024)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion](https://mlsanity.com/paper/2311.06318/) (2023)
- [Developing Effective Educational Chatbots with ChatGPT prompts: Insights from Preliminary Tests in a Case Study on Social Media Literacy (with appendix)](https://mlsanity.com/paper/2306.10645/) (2023)
- [Learning to Prompt in the Classroom to Understand AI Limits: A pilot study](https://mlsanity.com/paper/2307.01540/) (2023)
- [Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation](https://mlsanity.com/paper/2405.06681/) (2024)
- [Battling Botpoop using GenAI for Higher Education: A Study of a Retrieval Augmented Generation Chatbots Impact on Learning](https://mlsanity.com/paper/2406.07796/) (2024)

## Built on by (most cited first, 4 of 4)

- [Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines](https://mlsanity.com/paper/2504.07840/) (2025)
- [Geolog-IA: Conversational System for Academic Theses](https://mlsanity.com/paper/2510.02653/) (2025)
- [The Personalization Paradox: Semantic Loss vs. Reasoning Gains in Agentic AI Q&A](https://mlsanity.com/paper/2512.04343/) (2025)
- [Beyond Benchmark Scores: How Synthetic and Authentic Query Distributions Diverge in RAG Evaluation](https://mlsanity.com/paper/2609.14579/) (2026)

## Abstract (arXiv)

Natural language processing skills of Large Language Models (LLMs) are unprecedented, having wide diffusion and application in different tasks. This pilot study focuses on specializing ChatGPT behavior through a Retrieval-Augmented Generation (RAG) system using the OpenAI custom GPTs feature. The purpose of our chatbot, called Unimib Assistant, is to provide information and solutions to the specific needs of University of Milano-Bicocca (Unimib) students through a question-answering approach. We provided the system with a prompt highlighting its specific purpose and behavior, as well as university-related documents and links obtained from an initial need-finding phase, interviewing six students. After a preliminary customization phase, a qualitative usability test was conducted with six other students to identify the strengths and weaknesses of the chatbot, with the goal of improving it in a subsequent redesign phase. While the chatbot was appreciated for its user-friendly experience, perceived general reliability, well-structured responses, and conversational tone, several significant technical and functional limitations emerged. In particular, the satisfaction and overall experience of the users was impaired by the system's inability to always provide fully accurate information. Moreover, it would often neglect to report relevant information even if present in the materials uploaded and prompt given. Furthermore, it sometimes generated unclickable links, undermining its trustworthiness, since providing the source of information was an important aspect for our users. Further in-depth studies and feedback from other users as well as implementation iterations are planned to refine our Unimib Assistant.