Research on Fine-Tuning Large Language Models for Teaching Assistance in Data Science based on LoRA/QLoRA

Shaobo Xu; Pengqi Duan; Zhang Feng

doi:10.6919/ICJE.202606_12(6).0002

Authors

Shaobo Xu
Pengqi Duan
Zhang Feng

DOI:

https://doi.org/10.6919/ICJE.202606_12(6).0002

Keywords:

Parameter-Efficient Fine-Tuning; Teaching Assistance; LoRA; QloRA.

Abstract

This paper compares LoRA and QLoRA for structured teaching assistance tasks in data science education under limited computing resources. A dataset with three tasks was built from student questionnaire data. A post-processing pipeline and evaluation metrics were designed for structured output. Experiments show that LoRA is better in training efficiency, while QLoRA performs better in JSON validity, field coverage, structure quality, and citation generation. Overall, QLoRA is more suitable for structured teaching assistance tasks in resource-constrained settings.

Downloads

Download data is not yet available.

References

[1] Ding, N., Qin, Y., Yang, G., et al. (2023). Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3), 220–235. https://doi.org/10.1038/s42256-023-00647-9

[2] Han, Z., Gao, C., Liu, J., et al. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608.

[3] Anisuzzaman, D. M., Malins, J. G., Friedman, P. A., et al. (2025). Fine-tuning large language models for specialized use cases. Mayo Clinic Proceedings: Digital Health, 3(1), 100184. https://doi.org/10.1016/j.mcpdig.2024.100184

[4] Tinn, R., Cheng, H., Gu, Y., et al. (2023). Fine-tuning large neural language models for biomedical natural language processing. Patterns, 4(4). https://doi.org/10.1016/j.patter.2023.100700

[5] Patil, R., Khot, P., & Gudivada, V. (2025). Analyzing LLAMA3 performance on classification task using LoRA and QLoRA techniques. Applied Sciences, 15(6), 3087. https://doi.org/10.3390/app15063087

[6] Susilo, A., Christanti, V., & Lauro, M. D. (2024). Fine-Tuning LLaMA-2-Chat untuk ChatBot Penerjemah Bahasa Gaul menggunakan LoRA dan QLoRA. MIND (Multimedia Artificial Intelligent Networking Database) Journal, 9(2), 248–260.

[7] Zhang, D., Feng, T., Xue, L., et al. (2025). Parameter-efficient fine-tuning for foundation models. arXiv preprint arXiv:2501.13787.

[8] Xu, L., Xie, H., Qin, S. J., et al. (2026). Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9] Chu, Z., Wang, S., Xie, J., et al. (2025). Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733.

[10] Wang, S., Chen, F., Chen, G., et al. (2025). Using Local LLM Tools to Optimize Chinese High School Chemistry Education: Practice, Challenges, and Future Directions. Journal of Chemical Education, 102(10), 4368–4375. https://doi.org/10.1021/acs.jchemed.5c00359

[11] Hsieh, P. J., Chen, C. C., & Liu, W. (2019). Integrating talent cultivation tools to enact a knowledge-oriented culture and achieve organizational talent cultivation strategies. Knowledge Management Research & Practice, 17(1), 108–124. https://doi.org/10.1080/14778238.2018.1548762

[12] Harry, A., & Sayudin, S. (2023). Role of AI in Education. Interdisciplinary Journal and Humanity (INJURITY), 2(3), 260–268.

[13] Beck, J., Stern, M., & Haugsjaa, E. (1996). Applications of AI in Education. XRDS: Crossroads, The ACM Magazine for Students, 3(1), 11–15.