ChatMIT: A University Recruitment Chatbot based on zero-shot self–retrieval augmented generation
Artykuł w czasopiśmie
MNiSW
100
Lista 2024
| Status: | |
| Autorzy: | Błaszczykowski Michał, Jastrzębska Magdalena, Paśnikowska-Łukaszuk Magdalena |
| Dyscypliny: | |
| Aby zobaczyć szczegóły należy się zalogować. | |
| Rok wydania: | 2026 |
| Wersja dokumentu: | Drukowana | Elektroniczna |
| Język: | angielski |
| Numer czasopisma: | 11 |
| Wolumen/Tom: | 20 |
| Strony: | 178 - 192 |
| Impact Factor: | 1,5 |
| Web of Science® Times Cited: | 0 |
| Scopus® Cytowania: | 0 |
| Bazy: | Web of Science | Scopus | BazTech |
| Efekt badań statutowych | NIE |
| Materiał konferencyjny: | NIE |
| Publikacja OA: | TAK |
| Licencja: | |
| Sposób udostępnienia: | Otwarte czasopismo |
| Wersja tekstu: | Ostateczna wersja opublikowana |
| Czas opublikowania: | W momencie opublikowania |
| Data opublikowania w OA: | 1 października 2026 |
| Abstrakty: | angielski |
| This study presents ChatMIT, a domain-specific conversational system designed to support undergraduate recruitment at the Faculty of Mathematics and Information Technology at Lublin University of Technology. Three Retrieval-Augmented Generation architectures, namely Naive RAG, Hybrid RAG, and Self-RAG, were implemented and evaluated across three levels of question complexity using three large language models of varying parameter scales: Gemma 3 (27B), Llama 3 (70B), and gpt-oss (120B). Response quality was assessed by independent domain experts according to two criteria: factual correctness and linguistic quality, each evaluated using a five-point scale developed by the authors. The results indicate that the choice of RAG architecture is the primary factor influencing response quality, exerting a greater impact than model scale. Self-RAG substantially outperformed both Hybrid RAG and Naive RAG across all levels of question complexity, achieving mean factual correctness scores of 4.67/5, 2.77/5, and 2.33/5, as well as linguistic quality scores of 4.90/5, 4.37/5, and 4.33/5, respectively. Furthermore, the adoption of a zero-shot variant of Self-RAG constitutes a methodological contribution of this study, demonstrating that the adaptive properties of the original architecture can be preserved while significantly reducing deployment complexity, without the need for supervised training of a dedicated critic model. The findings confirm that RAG-based conversational systems represent an effective approach to automating information access in academic environments and offer considerable potential for deployment across diverse institutional domains and faculties. |
