Efficient Few-Shot Classification of Indonesian Academic Correspondence Using SetFit
Keywords:
Academic Correspondence, Few-Shot Learning, SetFit, Text Classification, Natural Language Processing.Abstract
Digital transformation in higher education has increased the need for intelligent systems capable of organizing and classifying institutional correspondence efficiently, particularly when labeled training data are limited. This study develops and evaluates an Indonesian academic-correspondence classification system using SetFit within an efficient few-shot learning framework. A quantitative experimental design was applied to 145 official PDF documents collected from five units at Universitas Islam Sultan Agung. Native digital documents were extracted using PyMuPDF, whereas scanned documents were processed using Doctr OCR, followed by correspondence-specific preprocessing, manual ground-truth labeling, and structured dataset preparation. Model training employed intfloat/multilingual-e5-base with incremental N-shot experiments from 1-shot to 6-shot and contrastive representation learning using CosineSimilarityLoss. The 6-shot configuration, using six training documents per class, produced the best reported performance, achieving an accuracy of 85.87% and a weighted F1-score of 0.85. Training-loss and error-rate patterns also indicated stable model adaptation under the reported experimental conditions. The selected model was subsequently integrated into a lightweight Streamlit prototype. The findings demonstrate that SetFit provides an efficient approach for academic-document classification when institution-specific labeled datasets are scarce, while consistent label definition and real-time OCR integration remain important considerations for further system development.
Downloads
References
Amatriain, X., Sankar, A., Bing, J., Bodigutla, P. K., Hazen, T. J., & Kazi, M. (2023). Transformer models: an introduction and catalog. arXiv preprint arXiv:2302.07730. http://arxiv.org/abs/2302.07730
Aminah, S., & Aminah, S. (2024). Perbandingan Keakuratan Sistem Rekomendasi Produk Berbasis Content-Based Filtering Dan Collaborative Filtering Pada E-Commerce Shopee Menggunakan Matrik Precision, Recall Dan F1-Score. Online Repository of Universitas NU Kalimantan Selatan, 1-18. https://repository.unukase.ac.id/id/eprint/268/
Anwar, M. T., Permana, A. K., Ambarwati, L., & Agustin, D. (2021). Analyzing Public Opinion Based on Emotion Labeling Using Transformers. 2021 2nd International Conference on Innovative and Creative Information Technology, ICITech 2021, 74-78. https://doi.org/10.1109/ICITech50181.2021.9590110
Cahyawijaya, S., Winata, G. I., Wilie, B., Vincentio, K., Li, X., Kuncoro, A., Ruder, S., Lim, Z. Y., Bahar, S., Khodra, M. L., Purwarianti, A., & Fung, P. (2021). IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation. EMNLP 2021 - 2021 Conference on Empirical Methods in Natural Language Processing, Proceedings, 8875-8898. https://doi.org/10.18653/v1/2021.emnlp-main.699
Juarto, B. ., & Yulianto. (2023). Indonesian News Classification Using IndoBert . International Journal of Intelligent Systems and Applications in Engineering, 11(2), 454–460. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/2654
Khaled, A., & Manayer, A. (2024). Few-shot Learning Approach for Arabic Scholarly Paper Classification using SetFit Framework. Wseas Transactions on Communications, 23, 89-95. https://doi.org/10.37394/23204.2024.23.12
Kilic, B., Bex, F., & Gatt, A. (2023). Contrast is all you need. arXiv preprint arXiv:2307.02882. https://doi.org/10.48550/arXiv.2307.02882
Koto, F., Rahimi, A., Lau, J. H., & Baldwin, T. (2020). IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP. COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference, 757-770. https://doi.org/10.18653/v1/2020.coling-main.66
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022). A survey of transformers. AI Open, 3(1), 111-132. https://doi.org/10.1016/j.aiopen.2022.10.001
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2021). Deep learning--based text classification: a comprehensive review. ACM computing surveys (CSUR), 54(3), 1-40. https://doi.org/10.1145/3439726
Molina, M., & Sanchez, J. (2022). Performance Variability in Akademik Classification. NeurIPS, 7-9. https://doi.org/10.52591/lxai202012123
Mursyidah, M., Davi, M., & Novitri, S. D. (2024). Klasifikasi Sentimen Review Pengguna terhadap Aplikasi Instagram menggunakan Algoritma Random Forest. Jurnal Infomedia: Teknik Informatika, Multimedia, dan Jaringan, 9(2), 106-115. http://dx.doi.org/10.30811/jim.v9i2.6069
Nabiilah, G. Z., Alam, I. N., Purwanto, E. S., & Hidayat, M. F. (2024). Indonesian multilabel classification using IndoBERT embedding and MBERT classification. International Journal of Electrical and Computer Engineering, 14(1), 1071-1078. https://doi.org/10.11591/ijece.v14i1.pp1071-1078
Nugroho, Y. A., Widodo, A., Pebrina, E. T., Iskandar, J., & Nadeak, M. (2025). Digitalization in Higher Education: How Information Systems Improve Operational and Strategic Performance. Indonesian Journal of Management and Economic Research (IJOMER), 2(01), 90-98. https://doi.org/10.70508/dyrdm592
Parnami, A., & Lee, M. (2022). Learning from few examples: A summary of approaches to few-shot learning. arXiv preprint arXiv:2203.04291. https://doi.org/10.48550/arXiv.2203.04291
Song, Y., Upadhyay, S., Peng, H., Mayhew, S., & Roth, D. (2019). Toward any-language zero-shot topic classification of textual documents. Artificial Intelligence, 274, 133-150. https://doi.org/10.1016/j.artint.2019.02.002
Syazali, M. R., & Yulianti, E. (2025). Classification of Economic Activities in Indonesia Using IndoBERT Language Model. Jurnal Ilmu Komputer dan Informasi, 18(2), 155-165. https://doi.org/10.21609/jiki.v18i2.1446
Tunstall, L., Reimers, N., Jo, U. E. S., Bates, L., Korat, D., Wasserblat, M., & Pereg, O. (2022). Efficient few-shot learning without prompts. arXiv preprint arXiv:2209.11055. http://arxiv.org/abs/2209.11055
Vergni, L. (2021). Evaluasi kesamaan antara indeks kekeringan melalui analisis korelasi dan uji Kappa Cohen di wilayah Mediterania. Nat Hazards 108, 2187–2209 (2021). https://doi.org/10.1007/s11069-021-04775-w
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021). Generalizing from a Few Examples: A Survey on Few-shot Learning. ACM Computing Surveys, 53(3), 1-34. https://doi.org/10.1145/3386252
Zaplana, I., Hadfield, H., & Lasenby, J. (2022). Closed-form solutions for the inverse kinematics of serial robots using conformal geometric algebra. Mechanism and Machine Theory, 173. https://doi.org/10.1016/j.mechmachtheory.2022.104835
Zhou, M., Duan, N., Liu, S., & Shum, H. Y. (2020). Progress in Neural NLP: Modeling, Learning, and Reasoning. Engineering, 6(3), 275-290. https://doi.org/10.1016/j.eng.2019.12.014
