Content Validity of Three-Tier Multiple Choice Virtual Reality-Based Assessment with Embedded Ethno-SSI Concepts on Acid-Base Topics
DOI:
https://doi.org/10.58524/oler.v5i1.665Keywords:
Content validity, Ethno-socio-scientific issue, Misconceptions, Three-tier multiple choice, Virtual reality based assesmentAbstract
Research on acid-base misconceptions at the high school level is extensive, but studies at the university level remain limited. Due to the prevalence of these misconceptions, developing accurate instruments for early detection is crucial. This study developed a three-tier multiple-choice virtual reality-based assessment integrating the ETHNO-SSI concept for acid-base topics. The study employed the Borg and Gall R and D model to evaluate content validity.The study used a developmental design, with face validation by six expert reviewers using a rubric to evaluate content validity, construct relevance, clarity and contextual integration. Processing of the data involved the use of Aiken’s V index. The results also evidenced a high content validity for all the dimensions studied, indicating that the instrument is pedagogically sound and culturally relevant. By embedding Ethno-SSI in immersive VR platforms, this assessment approaches represents a new path in the pursuit of students’ conceptual challenges when dealing with sociocultural decision making in real-world conditions. This study provides valuable information to the development of multidimensional chemical assessment instruments to link sociocultural innovation, recover notions of acid-base misconceptions, and incorporate cultural relevance in chemistry education.References
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Arslan, H. O., Cigdemoglu, C., & Moseley, C. (2012). A three-tier diagnostic test to assess pre-service teachers’ misconceptions about global warming, greenhouse effect, ozone layer depletion, and acid rain. International Journal of Science Education, 34(11), 1667–1686. https://doi.org/10.1080/09500693.2012.680618
Azamat, J., Khodadust, M. R., & Bahrami Maddah, A. M. (2024). The role of technical english proficiency in chemistry education. Chemical Review and Letters, 7(8), 731–741.
Banawi, A., Sopandi, W., Kadarohman, A., & Solehuddin, M. (2022). Five-tier multiple-choice diagnostic test development: Empirical evidences to improve students’ science literacy. Proceedings of the International Conference on Madrasah Reform, 633(1), 131–138. https://doi.org/10.2991/assehr.k.220104.020
Barke, Hazari, A., & Yitbarek, S. (2009). Misconceptions in chemistry. In Misconceptions in Chemistry. Springer Berlin Heidelberg, 15-20. https://doi.org/10.1007/978-3-540-70989-3
Borg, W. R., & Gall, M. D. (1983). Educational Research: An Introduction. Longman.
Cetin-Dindar, A., & Geban, O. (2011). Development of a three-tier test to assess high school students’ understanding of acids and bases. Procedia - Social and Behavioral Sciences, 15, 600–604. https://doi.org/10.1016/j.sbspro.2011.03.147
Chen, C.-W., Andersson, B., & Zhu, J. (2023). A factor mixture model for item responses and certainty of response indices to identify student knowledge profiles. Journal of Educational Measurement, 60(1), 28–51. https://doi.org/10.1111/jedm.12344
Chen, L., Hui, C., Qun, H., Jianbo, X., & and Teng, H. (2022). Absorption, metabolism and bioavailability of flavonoids: a review. Critical Reviews in Food Science and Nutrition, 62(28), 7730–7742. https://doi.org/10.1080/10408398.2021.1917508
Coduto, J. R., Lazicki, A., & Leddy, J. (2024). Visualizing 3D objects in analytical chemistry. Journal of Chemical Education, 101(1), 77–87. https://doi.org/10.1021/acs.jchemed.3c00821
Di Natale, A. F., Repetto, C., Riva, G., & Villani, D. (2020). Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. British Journal of Educational Technology, 51(6), 2006–2033. https://doi.org/10.1111/bjet.13030
Diani, R., Alfin, J., Anggraeni, Y. M., Mustari, M., & Fujiani, D. (2019). Four-Tier diagnostic test with certainty of response index on the concepts of fluid. Journal of Physics: Conference Series, 1155(1). https://doi.org/10.1088/1742-6596/1155/1/012078
Dood, A. J., Fields, K. B., & Raker, J. R. (2018). Using lexical analysis to predict lewis acid-base model use in responses to an acid-base proton-transfer reaction. Journal of Chemical Education, 95(8), 1267–1275. https://doi.org/10.1021/acs.jchemed.8b00177
Duca, A., Constantinescu, G.-G., & Iftene, A. (2024). Future education: Experimenting with chemical reactions in virtual reality. 18th International Conference on INnovations in Intelligent SysTems and Applications, INISTA 2024. Institute of Electrical and Electronics Engineers Inc, 1(1), 1-6. https://doi.org/10.1109/INISTA62901.2024.10683861
Elford, D., Lancaster, S. J., & Jones, G. A. (2022). Fostering motivation toward chemistry through augmented reality educational escape activities. A self-determination theory approach. Journal of Chemical Education, 99(10), 3406–3417. https://doi.org/10.1021/acs.jchemed.2c00428
Febu, R., Sudarmin, M, N., & W, S. (2016). Development of ethnoscience approach in the module theme substance additives to improve the cognitive learning outcome and student’s entrepreneurship. Journal of Physics: Conference Series, 755(1). https://doi.org/10.1088/1742-6596/755/1/011001
Finken, J., & Wölfel, M. (2023). Influence of immersive virtual reality on cognitive and affective learning goals. Lecture Notes in Networks and Systems, 581(1), 510–521. https://doi.org/10.1007/978-3-031-21569-8_48
Gkitzia, V., Salta, K., & Tzougraki, C. (2020). Students’ competence in translating between different types of chemical representations. Chemistry Education Research and Practice, 21(1), 307–330. https://doi.org/10.1039/C8RP00301G
Gulacar, O., Milkey, A., & Eilks, I. (2020). Exploring cluster changes in students’ knowledge structures throughout general chemistry. Eurasia Journal of Mathematics, Science and Technology Education, 16(6), 1850. https://doi.org/10.29333/EJMSTE/7860
Guruloo, T. N. M., & Osman, K. (2023). Integrating virtual reality laboratories (VRLs) in chemistry education: A systematic literature review. International Journal of Education, 15(4), 127. https://doi.org/10.5296/ije.v15i4.21372
Hadinugrahaningsih, T., Ridwan, A., Rahmawati, Y., Allanas, E., Cahya N., G., & Amalia, R. (2021). An analysis of chemistry student’s laboratory jargon in acid-base material using a 3E learning cycle. In M. M., R. Y., D. M., & F. E. (Eds.), AIP Conf. Proc. 2 April 2021; 2331 (1): 040035. https://doi.org/10.1063/5.0045512
Hakimah, N., Muchson, M., Herunata, H., Permatasari, M. B., & Santoso, A. (2021). Identification student misconceptions on reaction rate using a Google forms three-tier tests. In S. H., H. H., & R. D. (Eds.), AIP Conf. Proc. 2 March 2021; 2330 (1): 020020. https://doi.org/10.1063/5.0043114
Hammer, M., & Avram, E. M. G. (2024). Online interactive activity: Using a web-based multimedia activity to teach balancing chemical equations. Journal of Chemical Education, 101(10), 4510-4516. https://doi.org/10.1021/acs.jchemed.4c00786
Hendry, R. F. (2016). Structure as abstraction. Philosophy of Science, 83(5), 1070–1081. https://doi.org/10.1086/687939
Herunata, H., Wijaya, I., Sulistina, O., & Nazriati, N. (2024). The development of teaching materials based on conceptual understanding, chemical representation, and representational competence in chemical kinetics. In H. H. & R. T. (Eds.), AIP Conference Proceedings, American Institute of Physics, 3106, (1), 040014. https://doi.org/10.1063/5.0214803
Hoai, V. T. T., Son, P. N., An, D. T. T., & Anh, N. V. (2024). An investigation into whether applying augmented reality (ar) in teaching chemistry enhances chemical cognitive ability. International Journal of Learning, Teaching and Educational Research, 23(4), 195–216. https://doi.org/10.26803/ijlter.23.4.11
Jiang, G., Xia, X., Li, Y., Liang, H.-N., & Hui, P. (2024). ChemistryVR: Enhancing educational experiences through virtual chemistry lab simulations. In S. S.N. (Ed.), Proceedings - SIGGRAPH Asia 2024 Educator’s Forum, SA 2024. Association for Computing Machinery, Inc. 1(1), 1-5 https://doi.org/10.1145/3680533.3697068
Jiménez-Liso, M. R., López-Banet, L., & Dillon, J. (2020). Changing how we teach acid-base chemistry: A proposal grounded in studies of the history and nature of science education. Science and Education, 29(5), 1291–1315. https://doi.org/10.1007/s11191-020-00142-6
Johnson, M. D., Lavner, J. A., Mund, M., Zemp, M., Stanley, S. M., Neyer, F. J., Impett, E. A., Rhoades, G. K., Bodenmann, G., Weidmann, R., Bühler, J. L., Burriss, R. P., Wünsche, J., & Grob, A. (2022). Clinical psychology: Science and practice commentary. Personality and Social Psychology Bulletin, 48(4), 534–549. https://doi.org/10.1177/01461672211016920
Johnstone, A. H., Macdonald, J. J., & G Webb. (1977). Misconceptions in school thermodynamics. Physics Education, 12(4), 248. https://doi.org/10.1088/0031-9120/12/4/011
Julaeha, S., Hidayat, T., & Rustaman, N. Y. (2020). Development of web-based three tier multiple choice test to measure student’s tree thinking; Try out. Journal of Physics: Conference Series, 1521(4), 39-40. https://doi.org/10.1088/1742-6596/1521/4/042024
Kala, N., Yaman, F., & Ayas, A. (2013). The effectiveness of predict-observe-explain technique in probing students’ understanding about acid-base chemistry: A case for the concepts of ph, poh, and strength. International Journal of Science and Mathematics Education, 11(3), 555–574. https://doi.org/10.1007/s10763-012-9354-z
Kean, E., & Middlecamp, K. (1985). Panduan belajar kimia dasar. Erlangga.
Keiner, L., & Graulich, N. (2021). Beyond the beaker: Students’ use of a scaffold to connect observations with the particle level in the organic chemistry laboratory. Chemistry Education Research and Practice, 22(1), 146–163. https://doi.org/10.1039/D0RP00206B
Kounlaxay, K., Yao, D., Ha, M. W., & Kim, S. K. (2022). Design of virtual reality system for organic chemistry. Intelligent Automation and Soft Computing, 31(2), 1119–1130. https://doi.org/10.32604/iasc.2022.020151
Krajčovič, M., Gabajová, G., Matys, M., Grznár, P., Dulina, Ľ., & Kohár, R. (2021). 3D Interactive learning environment as a tool for knowledge transfer and retention. Sustainability (Switzerland), 13(14), 1–23. https://doi.org/10.3390/su13147916
Kurniawan, W., & Basuki, F. R. (2024). Ethnoscience learning: How do teacher implementing to increase scientific literacy in junior high school. International Journal of Evaluation and Research in Education (IJERE), 13, 1719. https://doi.org/10.11591/ijere.v13i3.26180
Liu, D., Bhagat, K., Yuan, G., Huang, R., & Chang, T. (2017). The potentials and trends of virtual reality in education. 105–130. https://doi.org/10.1007/978-981-10-5490-7_7
Makransky, G., & Lilleholt, L. (2018). A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educational Technology Research and Development, 66(5), 1141–1164. https://doi.org/10.1007/s11423-018-9581-2
Maksimenko, N., Okolzina, A., Vlasova, A., Tracey, C., & Kurushkin, M. (2021). Introducing atomic structure to first-year undergraduate chemistry students with an immersive virtual reality experience. Journal of Chemical Education, 98(6), 2104–2108. https://doi.org/10.1021/acs.jchemed.0c01441
Mubarak, S., & Yahdi. (2020). Identifying undergraduate students’ misconceptions in understanding acid base materials. Jurnal Pendidikan IPA Indonesia, 9(2), 276–286. https://doi.org/10.15294/jpii.v9i2.23193
Muyassaroh, I., Amiroh, A., Maryadi, M., & Masruroh, N. (2016). Integrasi kearifan lokal dalam kurikulum sains di sekolah dasar: Tinjauan literatur sistematis. Kalam Cendikia: Jurnal Ilmiah Kependidikan, 12, 1–23. https://doi.org/10.20961/jkc.v12i3.93360
Nahadi, N., Siswaningsih, W., Firman, H., Dewi, E. P., Lestari, T., & Rahmawati, T. (2023). Development and application of a two-tier acid-base misconception diagnostic test based on pictorial to identifying student misconceptions in chemistry. Journal of Engineering Science and Technology, 18(1), 207–223.
Nisa’, F. N., Widodo, W., Roqobih, & Dian, F. (2024). Pembelajaran inkuiri terbimbing untuk meningkatkan hasil belajar siswa pada materi pencemaran lingkungan. BIOCHEPHY: Journal of Science Education, 4(1), 330–336.
Osman, S. A., Razali, S. F. M., Shokri, S. N. S. M., Othman, A., Badaruzzaman, W. H. W., Taib, K. A., & Khoiry, M. A. (2016). Effectiveness of pre-Test in determining students’ achievement in department fundamental courses. Pertanika Journal of Social Sciences and Humanities, 24(1), 49–62.
Paristiowati, M., Zulmanelis, Z., & Nurhadi, M. F. (2019). Green chemistry-based experiments as the implementation of sustainable development values. JTK (Jurnal Tadris Kimiya), 4(1), 11–20. https://doi.org/10.15575/jtk.v4i1.3566
Rahayu, S. (2019). Socio-scientific Issues ( SSI ) in chemistry education : Enhancing both students ’ chemical literacy & transferable skills, 1227(1), 012008 https://doi.org/10.1088/1742-6596/1227/1/012008
Rashid, S., Khattak, A., Ashiq, M., Rehman, S. U., & Rasool, M. R. (2021). Educational landscape of virtual reality in higher education: Bibliometric evidences of publishing patterns and emerging trends. Publications, 9(2), 17. https://doi.org/10.3390/publications9020017
Reina, M., This, H., & Reina, A. (2022). Improving the understanding of chemistry by using the right words: A clear-cut strategy to avoid misconceptions when talking about elements, atoms, and molecules. Journal of Chemical Education, 99(8), 2999–3006. https://doi.org/10.1021/acs.jchemed.2c00411
Ristanto, R. H., Suryanda, A., & Indraswari, L. A. (2023). The development of ecosystem misconception diagnostic test. International Journal of Evaluation and Research in Education, 12(4), 2246–2259. https://doi.org/10.11591/ijere.v12i4.25200
Rodriguez, J.-M. G., Hunter, K. H., Scharlott, L. J., & Becker, N. (2020). A review of research on process oriented guided inquiry learning: Implications for research and practice. Journal of Chemical Education, 97(10), 3506–3520. https://doi.org/10.1021/acs.jchemed.0c00355
Romine, W. L., Todd, A. N., & Clark, T. B. (2016). How do undergraduate students conceptualize acid–base chemistry? measurement of a concept Progression. Science Education, 100(6), 1150–1183. https://doi.org/10.1002/sce.21240
Santos, V. C., & Arroio, A. (2016). The representational levels: Influences and contributions to research in chemical education. Journal of Turkish Science Education, 13(1), 3–18. https://doi.org/10.12973/tused.10153a
Shaafi, N. F., Yusof, M. M. M., Ellianawati, E., Subali, B., & Raji’e, M. H. H. (2025). Investigating misconceptions about acids and bases among pre-service science teachers. Journal of Education and Learning, 19(1), 460–477. https://doi.org/10.11591/edulearn.v19i1.21803
Sparks, B., Zidenberg, A. M., & Olver, M. E. (2022). Involuntary celibacy: A review of incel ideology and experiences with dating, rejection, and associated mental health and emotional sequelae. Current Psychiatry Reports, 24(12), 731–740. https://doi.org/10.1007/s11920-022-01382-9
Siswaningsih, W., & Chandratika, V. (2020). Profile of misconception in senior high school students on the concept of acid-base strength. Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar, MSCEIS 2019. European Alliance for Innovation. https://doi.org/10.4108/eai.12-10-2019.2296380
Stieff, M. (2019). Improving learning outcomes in secondary chemistry with visualization-supported inquiry activities. Journal of Chemical Education, 96(7), 1300–1307. https://doi.org/10.1021/acs.jchemed.9b00205
Sudarmin, S., Pujiastuti, R. S. E., Asyhar, R., Tri Prasetya, A., Diliarosta, S., & Ariyatun, A. (2023). Chemistry project-based learning for secondary metabolite course with ethno-STEM approach to improve students’ conservation and entrepreneurial character in the 21st century. Journal of Technology and Science Education, 13(1), 393. https://doi.org/10.3926/jotse.1792
Sumarni, W., Sumarti, S. S., & Kadarwati, S. (2023). Blended inquiry learning with ethno-stem approach for first-semester students’ chemical literacy. Jurnal Pendidikan IPA Indonesia, 12(3), 439–450. https://doi.org/10.15294/jpii.v12i3.45879
Viehmann, C., Fernández Cárdenas, J. M., & Reynaga Peña, C. G. (2024). The use of socioscientific issues in science lessons: a scoping review. Sustainability (Switzerland), 16(14), 5827. https://doi.org/10.3390/su16145827
Vlah, D., Čok, V., & Urbas, U. (2021). Vr as a 3d modelling tool in engineering design applications. Applied Sciences (Switzerland), 11(16), 7570. https://doi.org/10.3390/app11167570
Widarti, H. R., Wiyarsi, A., Yamtinah, S., & Shidiq, A. S. (2025). Analysis of content development in chemical materials related to ethnoscience : A review. Journal of Education and Learning (EduLearn), 19(1), 422–430. https://doi.org/10.11591/edulearn.v19i1.21210
Wu, Q., Vanerum, M., Agten, A., Christiansen, A., Vandenabeele, F., Rigo, J. M., & Janssen, R. (2021). Certainty-based marking on multiple-choice items: Psychometrics meets decision theory. Psychometrika, 86(2), 518–543. https://doi.org/10.1007/s11336-021-09759-0
Yamtinah, S., Susanti VH, E., Saputro, S., Ariani, S. R. D., Shidiq, A. S., Sari, D. R., & Ilyasa, D. G. (2023). Augmented reality learning media based on tetrahedral chemical representation: How effective in learning process? Eurasia Journal of Mathematics, Science and Technology Education, 19(8), 2313. https://doi.org/10.29333/ejmste/13436
Yang, H. M., & Hwang, S. Y. (2016). Reliability and validity of the assessment tool for measuring communication skills in nursing simulation education. Korean Journal of Adult Nursing, 28(1), 95-96. https://doi.org/10.7475/kjan.2016.28.1.95
Zamudio, J. O., Miguel-Gómez, J. E., Santiago, A., Montaño-Hilario, J. M., Franco-Bodek, D., García-Ortega, H., Reina, A., & Reina, M. (2024). Chemical element lotto: A captivating guessing and cultural game inspired by the mexican lottery. Journal of Chemical Education, 101(11), 4820-4829. https://doi.org/10.1021/acs.jchemed.4c00759
Zidny, R., & Eilks, I. (2022). Learning about pesticide use adapted from ethnoscience as a contribution to green and sustainable chemistry education. Education sciences, 12(4), 227. https://doi.org/10.3390/educsci12040227
Zidny, R., & Sjöström, J. (2021). A multi-perspective reflection on how indigenous knowledge and related ideas can improve science education for sustainability. 29, 145–185. https://doi.org/10.1007/s11191-019-00100-x
Downloads
Published
Issue
Section
License
Online Learning in Educational Research is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with Online Learning in Educational Research agree to the following terms:
Copyright Retention: Authors retain the copyright of their work without any restrictions.
Publishing Rights: Authors retain the right to publish and distribute their work without any restrictions.
License Agreement: By publishing with Online Learning in Educational Research, authors agree that their work will be licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA). This license allows others to share and adapt the work, provided that appropriate credit is given, any changes are indicated, and the new creations are licensed under the same terms.
