The Implementation of Artificial Intelligence (AI) Technology and Students' Digital Literacy on Learning Outcomes in the Economics Learning Evaluation Course

Supriyadi Supriyadi; Ayu Nurul Amalia

doi:10.58524/smartsociety.v6i1.1002

Authors

Supriyadi Supriyadi Universitas Panca Sakti Bekasi, Indonesia
Ayu Nurul Amalia Universitas Panca Sakti Bekasi, Indonesia

DOI:

https://doi.org/10.58524/smartsociety.v6i1.1002

Keywords:

Implementation of AI , Technology, Digital Literacy, Learning Outcomes in the Learning Evaluation Course

Abstract

The rapid development of information technology requires students to possess adequate digital literacy skills to effectively access, evaluate, and utilize information in the learning process. At the same time, the implementation of Artificial Intelligence (AI) in education offers significant potential to enhance learning effectiveness, efficiency, and personalization. Therefore, it is important to examine how these two factors contribute to students’ academic achievement. This study aims to analyze the effect of AI implementation and digital literacy on students’ learning outcomes in the Learning Evaluation course. This research employs a quantitative method with a survey approach. The population of the study consists of 90 fifth-semester students from the Economic Education Study Program. A sample of 79 students was determined using the Isaac and Michael table at a 1% significance level, and selected through random sampling techniques. Data were collected using research instruments that had been tested for validity and reliability to ensure accuracy and consistency. The data were analyzed using partial regression and multiple regression techniques to examine both individual and simultaneous effects of the independent variables on the dependent variable. The results of the study reveal that: (1) the implementation of AI has a positive and significant effect on students’ learning outcomes; (2) digital literacy also has a positive and significant effect on learning outcomes; and (3) the implementation of AI and digital literacy simultaneously have a positive and significant effect on students’ learning outcomes. These findings indicate that both technological integration and students’ competencies in utilizing digital resources play crucial roles in improving academic performance. This study contributes by providing empirical evidence regarding the influence of AI implementation and digital literacy on student learning outcomes. The findings can serve as a reference for educators and institutions in designing effective learning strategies, improving the quality of higher education, and supporting the achievement of Sustainable Development Goal 4 (SDG 4), which emphasizes inclusive and quality education for all.

References

Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020). Application and theory gaps during the rise of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 1(1), 1–20. https://doi.org/https://doi.org/10.1016/j.caeai.2020.100002

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage Publications, Inc.

Garzón, J., Patiño, E., & Marulanda, C. (2025). Systematic Review of Artificial Intelligence in Education: Trends, Benefits, and Challenges. In Multimodal Technologies and Interaction (Vol. 9, Issue 84, pp. 1–19). https://doi.org/10.3390/mti9080084

Gligorea, I., Cioca, M., Oancea, R., Gorski, A.-T., Gorski, H., & Tudorache, P. (2023). Adaptive Learning Using Artificial Intelligence in e-Learning : A Literature Review. Education Sciences, 13(1216), 1–27.

Hariyanto, H., Kristianingsih, F. X. D., & Rizqona, M. (2025). Artificial intelligence in adaptive education : a systematic review of techniques for personalized learning. Discover Education, 4(458), 1–18. https://doi.org/https://doi.org/10.1007/s44217-025-00908-6

Holmes, W., Porayska-pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., & Koedinger, K. R. (2022). Ethics of AI in Education : Towards a Community-Wide Framework. International Journal of Artificial Intelligence in Education, 32(1), 504–526.

Laar, E. Van, Deursen, A. J. A. M. Van, Dijk, J. A. G. M. Van, & Haan, J. De. (2017). The relation between 21st-century skills and digital skills : A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010

Marselina, S., Rukmin, A., Judijanto, L., Nurlaila, L., & Apriliyani, E. S. (2025). The Digital Literacy and Students ’ Analytical Abilities : Challenges and Solutions in the Implementation of Technology Based Learning. TOFEDU : The Future of Education Journal, 4(8), 4082–4090.

Nations, U. (2015). Goal 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. United Nations.

Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065–1078. https://doi.org/https://doi.org/10.1016/j.compedu.2012.04.016

Rohayati, Y., & Abdillah, A. (2024). Digital Transformation for Era Society 5.0 and Resilience: Urgent Issues from Indonesia. In Societies (Vol. 14, Issue 12, p. 266). https://doi.org/10.3390/soc14120266

Rojas, C. N., Peñafiel, G. A. A. P., Buitrago, D. F. L., & Romero, C. A. T. (2021). Society 5.0: A Japanese Concept for a Superintelligent Society. In Sustainability (Vol. 13, Issue 12, pp. 1–16). https://doi.org/10.3390/su13126567

Salas-pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial Intelligence and Learning Analytics in Teacher Education : A Systematic Review. Education Sciences, 12(569), 1–18.

Statistik, B. P. (2025). Telecommunication Statistics in Indonesia 2024.

Sugiyono. (2014). Metode Penelitian Kuantitatif, Kualitatif Dan R&D. Alfabeta.

Tinmaz, H., Lee, Y.-T., Fanea‑Ivanovici, M., & Baber, H. (2022). A systematic review on digital literacy. Smart Learning Environments, 9(21), 1–18. https://doi.org/10.1186/s40561-022-00204-y

Treve, M. (2024). Integrating Artificial Intelligence in Education : Impacts on Student Learning and Innovation. International Journal of Vocational Education and Training Research, 10(2), 61–69.

Umar, A. T., Ahsan, J., & Pratama, L. S. (2025). Pengaruh Pembelajaran Berdiferensiasi Terhadap Kemampuan Literasi Digital Mahasiswa Di Fakultas Ekonomi Universitas Negeri Medan Fakultas Ekonomi Universitas Negeri Medan. JKIP : Jurnal Kajian Ilmu Pendidikan, 6(4), 1556–1565.

UNESCO. (2023). Global Education Monitoring Report 2023: Technology in education — A tool on whose terms? https://doi.org/https://doi.org/10.54676/UZLO9240

Villiers, P. C. De. (2024). The Impact of Society 5 . 0 on Curriculum Development in Higher Education. Journal of Ethics in Highher Education, 4(1), 1–25. https://doi.org/10.26034/fr.jehe.2024.5953

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0

Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over ‑ reliance on AI dialogue systems on students ’ cognitive abilities : a systematic review. Smart Learning Environments, 11(28), 1–37. https://doi.org/10.1186/s40561-024-00316-7

Zhu, H., Sun, Y., & Yang, J. (2025). Towards responsible artificial intelligence in education: a systematic review on identifying and mitigating ethical risks. Humanities and Social Sciences Communications, 12(1111), 1–14.