Creative problem-solving assessment in fluid mechanics for senior high school students: Instrument validation and reliability analysis using the rasch model

Authors

  • Rahma Diani (Scopus ID: 57205348300) Universitas Lampung
  • Viyanti Viyanti Unila
  • Tri Jalmo Unila
  • Dewi Lengkana
  • Intan Erliana UIN Raden Intan Lampung

DOI:

https://doi.org/10.58524/jasme.v5i1.558

Keywords:

Assessment Instrument, Creative Problem-Solving, Physics Education, Rasch Model, Validity and Reliability.

Abstract

Background: Creative Problem-Solving (CPS) is a critical skill in physics education, enabling students to address real-world challenges through innovative and practical approaches. Despite its importance, there is a limited number of reliable and valid instruments specifically designed to assess CPS skills within the context of physics education.
Aim: This study aims to develop and prove the validation of a CPS assessment instrument using the Rasch model to ensure its reliability and accuracy in measuring CPS skills in physics education.
Method: The research followed the ADDIE development model, which includes analysis, design, development, implementation, and evaluation phases. The instrument, consisting of 20 essay items aligned with CPS indicators, was validated by three experts in physics education. It was then tested on 35 Grade XI students in Lampung Province. Data analysis was performed using the Rasch model through Winsteps software, focusing on fit analysis, reliability, item difficulty distribution, and dimensionality assessment.
Results: The instrument demonstrated strong internal consistency, confirming its reliability for assessing CPS skills in physics education. The construct validity was supported by fit analysis, indicating that most items functioned appropriately. However, some items exhibited potential bias and required revision. The dimensionality analysis confirmed that the instrument effectively measured a single underlying construct, ensuring its psychometric robustness. These findings suggest that the developed CPS assessment instrument is a reliable and valid tool for evaluating students’ creative problem-solving abilities in high school physics, particularly in fluid mechanics.
Conclusion: This study successfully developed and validated a CPS assessment instrument for high school physics education, demonstrating strong reliability and construct validity. The instrument effectively measures CPS skills, though some items require refinement to ensure fairness and accuracy. Future research should focus on further improving item quality and testing the instrument in diverse educational settings to enhance its applicability and generalizability.

References

Adams, W. K., & and Wieman, C. E. (2011). Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking. International Journal of Science Education, 33(9), 1289–1312. https://doi.org/10.1080/09500693.2010.512369

Agnezi, L. A. & Festiyed. (2023). Development of The Online Assessment Instrument for Fluid, Temperature, and Heat to measure the problem solving skills of high school students. Journal of Physics: Conference Series, 2582(1), 012053. https://doi.org/10.1088/1742-6596/2582/1/012053

Ambrosio, L., Rodríguez‐Blázquez, C., Ayala, A., & Forjaz, M. J. (2020). Rasch analysis of the living with chronic illness scale in Parkinson’s disease. BMC Neurology, 20(1). https://doi.org/10.1186/s12883-020-01926-7

Aytekin, A., & Topçu, M. S. (2024). Improving 6th Grade Students’ Creative Problem Solving Skills Through Plugged and Unplugged Computational Thinking Approaches. Journal of Science Education and Technology, 33(6), 867–891. https://doi.org/10.1007/s10956-024-10130-y

Baghaei, P. (2008). The Rasch model as a construct validation tool. Rasch Measurement Transactions, 22(1), 1145–1146.

Bao, L., & Koenig, K. (2019). Physics education research for 21st century learning. Disciplinary and Interdisciplinary Science Education Research, 1(1), 2. https://doi.org/10.1186/s43031-019-0007-8

Barbera, J., Naibert, N., Komperda, R., & Pentecost, T. C. (2021). Clarity on Cronbach’s Alpha Use. Journal of Chemical Education, 98(2), 257–258. https://doi.org/10.1021/acs.jchemed.0c00183

Bitzenbauer, P., Veith, J. M., Girnat, B., & Meyn, J.-P. (2022). Assessing Engineering Students’ Conceptual Understanding of Introductory Quantum Optics. Physics, 4(4), Article 4. https://doi.org/10.3390/physics4040077

Bond, T. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences, Third Edition (3rd ed.). Routledge. https://doi.org/10.4324/9781315814698

Boone, W. J., & Staver, J. R. (2020). Advances in Rasch Analyses in the Human Sciences. Springer International Publishing. https://doi.org/10.1007/978-3-030-43420-5

Branch, R. M. (2009). Instructional Design: The ADDIE Approach. Springer US. https://doi.org/10.1007/978-0-387-09506-6

Burkholder, E. W., Miles, J. K., Layden, T. J., Wang, K. D., Fritz, A. V., & Wieman, C. E. (2020). Template for teaching and assessment of problem solving in introductory physics. Physical Review Physics Education Research, 16(1), 010123. https://doi.org/10.1103/PhysRevPhysEducRes.16.010123

Chen, S. Y., Tsai, J. C., Liu, S. Y., & Chang, C. Y. (2021). The effect of a scientific board game on improving creative problem solving skills. Thinking Skills and Creativity, 41(June), 100921. https://doi.org/10.1016/j.tsc.2021.100921

Christensen, K. S., Oernboel, E., Nielsen, M. G., & Bech, P. (2019). Diagnosing depression in primary care: A Rasch analysis of the major depression inventory. Scandinavian Journal of Primary Health Care, 37(1). https://doi.org/10.1080/02813432.2019.1568703

DeHaan, R. L. (2009). Teaching Creativity and Inventive Problem Solving in Science. CBE—Life Sciences Education, 8(3), 172–181. https://doi.org/10.1187/cbe.08-12-0081

Diani, R., & Hartati, N. S. (2018). Flipbook berbasis literasi Islam: Pengembangan media pembelajaran fisika dengan 3D pageflip professional. Jurnal Inovasi Pendidikan IPA, 4(2), 234–244. https://doi.org/10.21831/jipi.v4i2.20819

Diani, R., Herliantari, Hesti, Irwandani, I., Saregar, A., & Umam, Rofiqul. (2019). Search, Solve, Create, and Share (SSCS) Learning Model: The Impact on the Students Creative Problem-Solving Ability on the Concept of Substance Pressure. 9(1), 65–77. https://doi.org/10.26740/jpfa.v9n1.p65-77

Elder, H. (2024). Te Waka Kuaka, Rasch Analysis of a Cultural Assessment Tool in Traumatic Brain Injury in Māori. Journal of Psychiatry and Psychiatric Disorders, 08(03). https://doi.org/10.26502/jppd.2572-519X0216

Farida, F., Alamsyah, Y. A., Anggoro, B. S., Andari, T., & Lusiana, R. (2024a). Rasch Measurement Validation of an Assessment Tool for Measuring Students’ Creative Problem-Solving through the Use of ICT. Pixel-Bit. Revista de Medios y Educación, 71, 83–106. https://doi.org/10.12795/pixelbit.107973

Farida, F., Alamsyah, Y. A., Anggoro, B. S., Andari, T., & Lusiana, R. (2024b). Rasch Measurement Validation of an Assessment Tool for Measuring Students’ Creative Problem-Solving through the Use of ICT. Pixel-Bit. Revista de Medios y Educación, 71, 83–106. https://doi.org/10.12795/pixelbit.107973

Fathonah, S., Cahyono, E., Iswari, R. S., Haryani, S., Sarwi, S., Lestari, N. H., & Kadarwati, S. (2023). Effects of Multirepresentation-Based Creative Problem-Solving Learning Model on Students’ Critical Thinking and Diet Nutritional Quality. Journal of Turkish Science Education, 20(4), 669–694. https://doi.org/10.36681/tused.2023.038

Fiteriani, I., Diani, R., Hamidah, A., & Anwar, C. (2021). Project-based learning through STEM approach: Is it effective to improve students’ creative problem-solving ability and metacognitive skills in physics learning? Journal of Physics: Conference Series, 1796(1), 012058. https://doi.org/10.1088/1742-6596/1796/1/012058

García-Carmona, A. (2023). Scientific Thinking and Critical Thinking in Science Education. Science &Amp; Education. https://doi.org/10.1007/s11191-023-00460-5

Huang, X., Ma, H., Wang, X., Zhong, J., Chen, Y., & Xu, M. (2023). Rasch analysis of the Hospital Anxiety and Depression Scale (HADS) in patients with major depression. Anales De Psicología, 39(3). https://doi.org/10.6018/analesps.550551

Islam, A. Y. M. A., Gu, X., Crook, C., & Spector, J. M. (2020). Assessment of ICT in Tertiary Education Applying Structural Equation Modeling and Rasch Model. Sage Open, 10(4). https://doi.org/10.1177/2158244020975409

Kalkbrenner, M. T. (2021). A Practical Guide to Instrument Development and Score Validation in the Social Sciences: The MEASURE Approach. Practical Assessment, Research & Evaluation, 26. https://eric.ed.gov/?id=EJ1311131

Kassiavera, S., Suparmi, A., Cari, C., & Sukarmin, S. (2024). Application of Rasch Model in Two-Tier Test for Assessing Critical Thinking in Physics Education. Journal of Baltic Science Education, 23(6), 1227–1242. https://doi.org/10.33225/jbse/24.23.1227

Li, X., Su, Z., Wang, L., Li, J., & Diao, Y. (2024). Psychometric evaluation of the pictorial scale of perceived movement skill competence in Chinese children: An item response theory analysis. Child: Care, Health and Development, 50(4), e13275. https://doi.org/10.1111/cch.13275

Linacre, J. (2002). A User’’s Guide to Winsteps: Rasch-Model Computer Program.

Maba, W., Perdata, I. B. K., & Astawa, I. N. (2017). Constructing Assessment Instrument Models for Teacher's Performance, Welfare and Education Quality. International Journal of Social Sciences and Humanities, 1(3), 88–96. https://doi.org/10.29332/ijssh.v1n3.59

Marfu’i, L. N. R., Ilfiandra, & Nurhudaya. (2019). The analysis of critical thinking skills test in social-problems for physics education students with Rasch Model. Journal of Physics: Conference Series, 1280(5), 052012. https://doi.org/10.1088/1742-6596/1280/5/052012

Maries, A., & Singh, C. (2023). Helping Students Become Proficient Problem Solvers Part I: A Brief Review. Education Sciences, 13(2), Article 2. https://doi.org/10.3390/educsci13020156

Medvedev, O. N., & Krägeloh, C. U. (2022). Rasch Measurement Model. In Handbook of Assessment in Mindfulness Research (pp. 1–18). Springer, Cham. https://doi.org/10.1007/978-3-030-77644-2_4-1

Mitchell, W. E., & Kowalik, T. F. (1979). Creative Problem Solving. Gifted Child Quarterly, 23(3), 559–561. https://doi.org/10.1177/001698627902300322

Naingalis, M. L. P., Fauzi, A., Supurwoko, & Sartono, B. (2023). Item fit investigation of computational thinking scales instrument among Indonesian physics undergraduate students: A Rasch model analysis. AIP Conference Proceedings, 2540(1), 090006. https://doi.org/10.1063/5.0105746

Park, Y.-J. (2021). Development of the 8-Item Phlegm Pattern Questionnaire (PPQ-8) Using Rasch Analysis. Evidence-Based Complementary and Alternative Medicine, 2021. https://doi.org/10.1155/2021/6528891

Pears, M., Antoniou, P., Schiza, E., Matsangidou, M., Pattichis, C. S., Bamidis, P. D., & Konstantinidis, Stathis. Th. (2025). Enhancing creativity and cognitive skills in healthcare curricula: Recommendations from a modified delphi study on virtual reality integration. Thinking Skills and Creativity, 57, 101810. https://doi.org/10.1016/j.tsc.2025.101810

Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111. https://doi.org/10.1103/PhysRevPhysEducRes.15.020111

Puccio, G. J., Klarman, B., & Szalay, P. A. (2022). Creative Problem-Solving. In The Palgrave Encyclopedia of the Possible (pp. 298–313). Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-90913-0_41

Purnami, W., Fauzi, A., & Naingalis, M. L. P. (2023). Computational thinking skills identification among students of physics education department using Rasch model analysis. AIP Conference Proceedings, 2751(1), 100012. https://doi.org/10.1063/5.0143158

Ringo, S. S., Kuswanto, K., Samsudin, A., & Setiawan, A. (2021). Rural and urban students’ attitudes toward physics: A comparative study using Rasch analysis. Journal of Physics: Conference Series, 1806(1), 012009. https://doi.org/10.1088/1742-6596/1806/1/012009

Robertson, R. L., & Larki, M. J. (2019). Developing an Instrument to Observe and Evaluate Assessment System Maturity. Journal of Educational Research and Practice, 9(1), 55–80. https://doi.org/10.5590/JERAP.2019.09.1.05

Shakhman, L., & Barak, M. (2019). The Physics Problem-Solving Taxonomy (PPST): Development and Application for Evaluating Student Learning. EURASIA Journal of Mathematics, Science and Technology Education, 15(11). https://doi.org/10.29333/ejmste/109266

Shin, Y., Lee, H., & Kim, W. (2025). Exploring creative problem-solving in computer-supported collaborative learning: Focusing on group cohesiveness and socially shared metacognitive regulation. Thinking Skills and Creativity, 56, 101771. https://doi.org/10.1016/j.tsc.2025.101771

Smith, G. T., & McCarthy, D. M. (1995). Methodological considerations in the refinement of clinical assessment instruments. Psychological Assessment, 7(3), 300–308. https://doi.org/10.1037/1040-3590.7.3.300

Song, Y., Huang, L., Zheng, L., Fan, M., & Liu, Z. (2025). Interactions with generative AI chatbots: Unveiling dialogic dynamics, students’ perceptions, and practical competencies in creative problem-solving. International Journal of Educational Technology in Higher Education, 22(1). Scopus. https://doi.org/10.1186/s41239-025-00508-2

Taber, K. S. (2018). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

Testa, I., Capasso, G., Colantonio, A., Galano, S., Marzoli, I., Scotti di Uccio, U., & Serroni, G. (2020). Validation of University Entrance Tests Through Rasch Analysis. In M. S. Khine (Ed.), Rasch Measurement: Applications in Quantitative Educational Research (pp. 99–124). Springer. https://doi.org/10.1007/978-981-15-1800-3_6

Treffinger, D. J. (1995). Creative Problem Solving: Overview and educational implications. Educational Psychology Review, 7(3), 301–312. https://doi.org/10.1007/BF02213375

Ulya, H., Sugiman, & Rosnawati, R. (2024). Design and content validity of mathematics creative problem-solving ability instrument for junior high school students. Eurasia Journal of Mathematics, Science and Technology Education, 20(6), em2462. https://doi.org/10.29333/ejmste/14661

Viyanti, V., Rosidin, U., & Shintya, R. E. (2022). Collaborative and Problem Solving Instruments in Project-Based Physics Learning | Indonesian Journal of Science and Mathematics Education. https://doi.org/10.24042/ijsme.v5i1.11636

Widyaningsih, S. W., Yusuf, I., Prasetyo, Z. K., & Istiyono, E. (2021). The Development of the HOTS Test of Physics Based on Modern Test Theory: Question Modeling through E-Learning of Moodle LMS. International Journal of Instruction, 14(4), 51–68. https://doi.org/10.29333/iji.2021.1444a

Xia, N., Haron, S. H., Huang, Y., & Niu, R. (2025). The effectiveness of CPS+SCAMPER teaching mode and strategies on student creativity. Thinking Skills and Creativity, 56, 101758. https://doi.org/10.1016/j.tsc.2025.101758

Ying, Y., & Tiemann, R. (2024). Development of an assessment tool for collaborative problem-solving skills in chemistry. Disciplinary and Interdisciplinary Science Education Research, 6(1), 25. https://doi.org/10.1186/s43031-024-00116-6

Yusuf, I., Widyaningsih, S. W., Prasetyo, Z. K., & Istiyono, E. (2020). Higher order thinking skills (HOTS)-oriented e-module in electric circuit. Journal of Physics: Conference Series, 1521(2), 022027. https://doi.org/10.1088/1742-6596/1521/2/022027

Downloads

Published

2025-06-03

Issue

Vol. 5 No. 1 (2025): Journal of Advanced Sciences and Mathematics Education

Section

Articles

License

Authors who publish with Journal of Advanced Sciences and Mathematics Education agree to the following terms:

  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).

Creative Commons License

Journal of Advanced Sciences and Mathematics Education is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.