Effectiveness of online judge system with problem-based learning for computational thinking improvement
DOI:
https://doi.org/10.17509/curricula.v5i1.95499Keywords:
computational thinking, HUSTOJ, online judge system, problem-based learningAbstract
Computational Thinking (CT) has become a crucial competence in 21st-century vocational education, particularly in programming-related fields. However, programming instruction in vocational high schools often emphasizes syntactic correctness rather than higher-order problem-solving processes. This study aims to analyze the effectiveness of integrating the HUTS Online Judge (HUSTOJ) system with a Problem-Based Learning (PBL) model in improving students’ CT skills and to examine students’ learning behavior patterns through system log data. The study employed a quantitative pre-experimental one-group pretest-posttest design involving 38 tenth-grade vocational students majoring in Software Engineering. CT skills were measured using Problem-Based programming tests covering decomposition, pattern recognition, abstraction, and algorithmic thinking, while learning behaviors were analyzed using HUSTOJ activity logs. The results indicate a significant improvement in CT skills, with a large effect size and a moderate-to-high normalized gain. Log analysis revealed distinct learning behavior profiles, highlighting the role of iterative refinement and persistence in CT development. These findings suggest that integrating HUSTOJ and PBL provides effective scaffolding for learning and process-oriented assessment, making it a promising instructional model for vocational programming education.
Abstrak
Computational Thinking (CT) merupakan kompetensi penting dalam pendidikan vokasi abad ke-21, khususnya pada pembelajaran pemrograman. Namun, praktik pembelajaran di sekolah menengah kejuruan masih cenderung berfokus pada ketepatan sintaks, sehingga kurang mendukung pengembangan proses berpikir tingkat tinggi. Penelitian ini bertujuan untuk menganalisis efektivitas integrasi HUTS Online Judge (HUSTOJ) system dengan model Problem-Based Learning (PBL) dalam meningkatkan kemampuan CT murid serta mengkaji pola perilaku belajar murid berdasarkan data log sistem. Penelitian ini menggunakan pendekatan kuantitatif dengan desain pre-eksperimental one-group pretest-posttest yang melibatkan 38 murid kelas X Rekayasa Perangkat Lunak. Kemampuan CT diukur melalui tes pemrograman berbasis masalah yang mencakup dekomposisi, pengenalan pola, abstraksi, dan pemikiran algoritmik, sedangkan perilaku belajar dianalisis melalui log aktivitas HUSTOJ. Hasil penelitian menunjukkan peningkatan CT yang signifikan dengan ukuran efek besar dan nilai N-Gain kategori sedang hingga tinggi. Analisis log mengungkap pola belajar yang menegaskan pentingnya proses iteratif dan ketekunan dalam pengembangan CT. Temuan ini menunjukkan bahwa integrasi HUSTOJ dan PBL mendukung pembelajaran berbasis proses dan asesmen autentik dalam pendidikan vokasi.
Kata Kunci: berpikir komputasional; HUSTOJ; online judge system; pembelajaran berbasis masalah
References
Bantu, A. F., Kozyra, A., & Wiora, J. (2025). Normality tests for transformed large measured data: a comprehensive analysis. Statistics in Transition New Series, 26(3), 195-208.
Brass, T., Kennedy, J., Gabriel, F., Neill, B., Devis, D., & Leonard, S. N. (2023). Learning analytics for lifelong career development: a framework to support sustainable formative assessment and self-reflection in programs developing career self-efficacy. Frontiers in Artificial Intelligence, 6(1), 1-10.
Castro-Alonso, J. C., de Koning, B. B., Fiorella, L., & Paas, F. (2021). Five strategies for optimizing instructional materials: instructor-and learner-managed cognitive load. Educational Psychology Review, 33(4), 1379-1407.
Demir, S. (2022). Comparison of normality tests in terms of sample sizes under different skewness and kurtosis coefficients. International Journal of Assessment Tools in Education, 9(2), 397-409.
Gasaymeh, A. M. M., & AlMohtadi, R. M. (2024). A scoping review of the literature on computational thinking in education in the Arab world. Eurasia Journal of Mathematics, Science and Technology Education, 20(11), 1-12.
Husaeni, D. F. A, Rahman, E. F., Mulyanti, B., Suherman, A., Abdullah, A. G., Riza, L. S., ... & Nugraha, E. (2025). Improving vocational students’ learning outcomes through problem-based learning with multimodal learning media. Curricula: Journal of Curriculum Development, 4(2), 1663-1678.
Ji, W., & Wong, G. K. W. (2025). Integrating problem-based learning and computational thinking: cultivating creative thinking in primary education. Frontiers in Education, 10(1), 1-20.
Jyrwa, J. J. R., Jayaraman, C., & Joseph, A. (2025). A comprehensive analysis on computational thinking in education: open issues and challenges. Engineering Proceedings, 107(1), 1-11.
Kafai, Y. B., & Proctor, C. (2022). A revaluation of computational thinking in K-12 education: moving toward computational literacies. Educational Researcher, 51(2), 146-151.
Khor, E. T., & K, M. (2023). A systematic review of the role of learning analytics in supporting personalized learning. Education Sciences, 14(1), 1-18.
Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 1-29.
Liu, Z., Gearty, Z., Richard, E., Orrill, C. H., Kayumova, S., & Balasubramanian, R. (2024). Bringing computational thinking into classrooms: a systematic review on supporting teachers in integrating computational thinking into K-12 classrooms. International Journal of STEM Education, 11(1), 1-18.
Mahanan, M. S., Talib, C. A., & Ibrahim, N. H. (2021). Online formative assessment in higher STEM education; a systematic literature review. Asian Journal of Assessment in Teaching and Learning, 11(1), 47-62.
Masiello, I., Mohseni, Z., Palma, F., Nordmark, S., Augustsson, H., & Rundquist, R. (2024). A current overview of the use of learning analytics dashboards. Education Sciences, 14(1), 1-15.
Mi, H., Liu, Y., Meng, R., Tang, Y., & Zhang, Q. (2024). A study of learning behavior analysis and student achievement based on an online judge system. Proceedings of the 2024 9th International Conference on Distance Education and Learning, 1(1), 278-284.
Moreno-Palma, N., Hinojo-Lucena, F. J., Romero-Rodríguez, J. M., & Cáceres-Reche, M. P. (2024). Effectiveness of problem-based learning in the unplugged computational thinking of university students. Education Sciences, 14(7), 1-12.
Navarrete, J., Giaconi, V., Contador, G., & Vazquez, M. (2024). Another reason why normalized gain should continue to be used to analyze concept inventories (and estimate learning rates). Arxiv Preprint, 1(1), 1-24.
Ngadengon, Z., Subramaniam, T. S., Yasak, Z., Syukri, M., & Hazim, M. N. (2024). Theory on computational thinking in education: a systematic review. International Journal of Education, Psychology and Counseling, 9(53), 488-507.
Nurulaisyah, D., Probosari, R. M., & Antrakusuma, B. (2024). Problem-based learning model on the collaboration and computational thinking skills. Innovations in Science Education and Practice, 1(1), 36-46.
Ottenbreit-Leftwich, A. T., Kwon, K., Brush, T. A., Karlin, M., Jeon, M., Jantaraweragul, K., Guo, M., Nadir, H., Gok, F., & Bhattacharya, P. (2021). The impact of an issue-centered problem-based learning curriculum on 6th grade girls’ understanding of and interest in computer science. Computers and Education Open, 2(1), 1-11.
Palop, B., Díaz, I., Rodriguez-Muniz, L. J., & Santaengracia, J. J. (2025). Redefining computational thinking: a holistic framework and its implications for K-12 education. Education and Information Technologies, 30(1), 1-26.
Rico-Juan, J. R., Sánchez-Cartagena, V. M., Valero-Mas, J. J., & Gallego, A. J. (2023). Identifying student profiles within online judge systems using explainable artificial intelligence. IEEE Transactions on Learning Technologies, 16(6), 955-969.
Santosa, E. B., & Sukmawati, F. (2024). Level of computational thinking and technological literacy skills to improve pre-service teacher learning innovation. Jurnal Kependidikan: Jurnal Hasil Penelitian dan Kajian Kepustakaan di Bidang Pendidikan, Pengajaran, dan Pembelajaran, 10(1), 338-352.
Sihombing, E. A. M., Ruslan, D., & Lubis, W. (2025). Analysis of differentiated learning with problem-based learning model to improve learning activity. Inovasi Kurikulum, 22(3), 1839-1852.
Susanti, M., Suyanto, S., Jailani, J., & Retnawati, H. (2023). Problem-based learning for improving problem-solving and critical thinking skills: a case on probability theory course. Journal of Education and Learning, 17(4), 507-525.
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: a systematic review of empirical studies. Computers and Education, 148, 1-64.
Tikva, C., & Tambouris, E. (2021). Mapping computational thinking through programming in K-12 education: a conceptual model based on a systematic literature Review. Computers and Education, 162(1), 1-33.
Villarroel, V., Melipillán, R., Santana, J., & Aguirre, D. (2024). How authentic are assessments in vocational education? An analysis from Chilean teachers, students, and examinations. Frontiers in Education, 9(1), 1-12.
Wang, J., Lin, P., Tang, Z., & Chen, S. (2023). How problem difficulty and order influence programming education outcomes in online judge systems. Heliyon, 9(11), 1-14.
Zhang, W., Guan, Y., & Hu, Z. (2024b). The efficacy of project-based learning in enhancing computational thinking among students: a meta-analysis of 31 experiments and quasi-experiments. Education and Information Technologies, 29(11), 14513-14545.
Zhu, Y., Zhao, Z., Tong, C., & Xia, X. (2020). Small private online judge: a new tool for empirical education research. Arxiv Preprint, 1(1), 1-5.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Agus Suratna Permadi, Muhamad Nursalman, Munir
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
