RANCANG BANGUN SISTEM RAG UNTUK MANAJEMEN DOKUMENTASI AKREDITASI PROGRAM STUDI
DOI:
https://doi.org/10.61722/jipm.v4i5.3074Keywords:
akreditasi program studi, Retrieval-Augmented Generation (RAG), fine-tuning, sme-embeddings, GPT-4o-mini, GLM-4.7-FlashAbstract
Proses akreditasi program studi melibatkan dokumen regulasi yang kompleks dan berjumlah besar, sehingga menyulitkan pengelola dalam mencari informasi secara cepat dan tepat. Penelitian ini bertujuan merancang dan membangun sistem Question-Answering (QA) berbasis Retrieval-Augmented Generation (RAG) untuk manajemen dokumentasi akreditasi Program Studi Ilmu Informatika di Universitas Katolik Darma Cendika (UKDC). Sistem mengintegrasikan 84 dokumen regulasi yang diekstraksi menggunakan PyMuPDF menjadi 4.613 potongan teks (chunk) dan diindeks ke dalam basis data vektor ChromaDB. Arsitektur RAG memanfaatkan model embedding BGE-M3 yang di-fine-tuning dengan metode sme-embeddings berbasis contrastive learning (MultipleNegativesRankingLoss) untuk mengoptimalkan pencarian dokumen spesifik domain, serta model bahasa besar (LLM) GPT-4o-mini sebagai generator jawaban akhir, dengan GLM-4.7-Flash sebagai model pembanding. Evaluasi modul retrieval menunjukkan peningkatan signifikan setelah fine-tuning, dengan Recall@1 naik dari 0,8100 menjadi 0,9300, NDCG@1 dari 0,8100 menjadi 0,9300, dan MRR@10 dari 0,8757 menjadi 0,9603. Evaluasi kualitas jawaban menggunakan BERTScore menunjukkan F1-Score GPT-4o-mini meningkat dari 72,85% menjadi 73,35%, unggul dibandingkan GLM-4.7-Flash yang hanya mencapai 63,85%. Pengujian usability dengan Nielsen's Heuristics oleh tiga dosen penguji memperoleh rata-rata skor 4,64 dari skala 5,00. Hasil ini mengonfirmasi bahwa sistem RAG yang dikembangkan layak dan efektif digunakan sebagai alat bantu administratif dalam proses akreditasi program studi.
References
AI, Z. (2026). GLM-4.7-Flash. Hugging Face. https://huggingface.co/zai-org/GLM-4.7-Flash
Bovas, A. P., Joy, M., Santo, N. C., Borde, S. R., & Deshpande, V. V. (2025). Navi: RAG-Powered LLM Chatbot for Academic Institutions. International Journal for Research in Applied Science and Engineering Technology, 13(11), 2339–2346. https://doi.org/10.22214/ijraset.2025.75655
Cui, J., Ning, M., Li, Z., Chen, B., Yan, Y., Li, H., Ling, B., Tian, Y., & Yuan, L. (2024). Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model. 1–11. http://arxiv.org/abs/2306.16092
Elkiran, H., & Rasheed, J. (2025). EvaRAG: Evaluating Advanced RAG Techniques With Indexing and Distance Metrics. IEEE Access, 13(December), 215724–215747. https://doi.org/10.1109/ACCESS.2025.3646665
Fajri, A., & Mandala, R. (2025). Optimizing Retrieval-Augmented Generation for Domain-Specific Knowledge Systems through Fine-Tuning and Prompt Engineering. 5(2), 344–350.
Forootani, A., Aliabadi, D. E., & Thraen, D. (2025). Bio-Eng-LMM AI Assist chatbot: A Comprehensive Tool for Research and Education. http://arxiv.org/abs/2409.07110
Husain, M. L., Wibisono, Y., & Anisyah, A. (2025). Development of an Academic Services Chatbot Based on Retrieval-Augmented Generation (RAG). Brilliance: Research of Artificial Intelligence, 5(2), 727–735. https://jurnal.itscience.org/index.php/brilliance/article/view/6719
Jc7k. (2026). sme-embeddings. https://github.com/jc7k/sme-embeddings
Mansour, Y. E. I., & Ibrahim, A. (2025). AI-Driven Framework for Academic Program Evaluation and Accreditation Standards. The Barcelona Conference on Education 2025: Official Conference Proceedings, 393–403. https://doi.org/10.22492/issn.2435-9467.2025.32
Moreno-Cediel, A., Garcia-Lopez, E., Garcia-Cabot, A., & De-Fitero-Dominguez, D. (2025). Optimising retrieval performance in RAG systems: A new growing window semantic chunking strategy to address weak semantic boundaries. Knowledge-Based Systems, 331(November), 114896. https://doi.org/10.1016/j.knosys.2025.114896
Noviyanti, & Salabi, A. S. (2026). Akreditasi dalam Perubahan Global: Isu Klasik, Dinamika Regulasi dan Teknologi, serta Strategi Adaptif. 9, 2880–2887.
Pilicita, A., & Barra, E. (2025). LLMs in Education: Evaluation GPT and BERT Models in Student Comment Classification. Multimodal Technologies and Interaction, 9(5). https://doi.org/10.3390/mti9050044
Putri, R. A., Nurramdhani, H. I., & Hartati, S. (2026). A Retrieval-Augmented Generation Framework for a Study Program Accreditation Question-Answering System. 16(2), 33375–33381.
Rotaru, O., Orhei, C., & Vasiu, R. (2026). Hybrid Usability Evaluation of an Automotive REM Tool: Human and LLM-Based Heuristic Assessment of IBM Doors Next. Applied Sciences, 16(2), 723. https://doi.org/10.3390/app16020723
Swacha, J., & Gracel, M. (2025). Retrieval-Augmented Generation (RAG) Chatbots for Education: A Survey of Applications. Applied Sciences (Switzerland), 15(8). https://doi.org/10.3390/app15084234
Tamm, Y. M., Damdinov, R., & Vasilev, A. (2021). Quality metrics in recommender systems: Do We calculate metrics consistently? In RecSys 2021 - 15th ACM Conference on Recommender Systems (Vol. 1, Issue 1). Association for Computing Machinery. https://doi.org/10.1145/3460231.3478848
Tang, G., Yousuf, O., & Jin, Z. (2024). Improving BERTScore for Machine Translation Evaluation Through Contrastive Learning. IEEE Access, 12, 77739–77749. https://doi.org/10.1109/ACCESS.2024.3406993
Yu, R., Wang, T., Liu, R., & Wang, Y. (2026). SF-RAG: Structure-Fidelity Retrieval-Augmented Generation for Academic Question Answering (Vol. 1, Issue 1). arXiv. http://arxiv.org/abs/2602.13647
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 JURNAL ILMIAH PENELITIAN MAHASISWA
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
