Psychometric evaluation of a statistical literacy instrument for pre-service mathematics teachers: Rasch analysis using R

Putri Nur Malasari; Siti Qomariyah; Savira Najwa Meilani

doi:10.58524/jasme.v6i3.1240

Authors

Putri Nur Malasari Universitas Islam Negeri Sunan Kudus, Indonesia
Siti Qomariyah Universitas Islam Negeri Sunan Kudus, Indonesia
Savira Najwa Meilani Universitas Islam Negeri Sunan Kudus, Indonesia

DOI:

https://doi.org/10.58524/jasme.v6i3.1240

Keywords:

Educational Measurement, Pre-Service Mathematics Teachers, R Rasch Model, Statistical Literacy

Abstract

Background: Statistical literacy is an essential competency for pre-service mathematics teachers, characterized by persistent challenges in interpreting and critically evaluating statistical information. Despite the development of several statistical literacy instruments, comprehensive Rasch-based psychometric evidence for instruments targeting pre-service mathematics teachers remains limited.

Purpose: This study aimed to evaluate the psychometric quality of a statistical literacy instrument for pre-service mathematics teachers using the Rasch measurement model implemented in R. The evaluation focused on item functioning, reliability, validity, and overall measurement characteristics.

Methods: Data were collected from 132 pre-service mathematics teachers enrolled in an education program. Rasch analysis was conducted to examine item fit and difficulty, reliability, dimensionality, local independence, and person-item targeting.

Results: The entire 15 items met the Rasch fit criteria (Infit and Outfit MNSQ = 0.5–1.5), thereby supporting the construct validity and internal consistency, with no misfitting items identified. Most items ranged from easy to moderate (−1.47 to +0.35 logits), with only a particular difficult item. Wright Map analysis showed that respondents' abilities generally exceeded the highest item difficulty, suggesting suboptimal targeting and reduced measurement precision for high-ability respondents.

Conclusions: The results showed a mild ceiling effect, with 40.2% of respondents exceeding the hardest item and 3.0% achieving perfect scores. Generally, the instrument exhibited adequate psychometric quality but required more difficult items to improve measurement precision for high-ability respondents. The script-based Rasch workflow in R also enhanced the transparency and reproducibility of psychometric evaluation.

References

Allen, M. S., Iliescu, D., & Greiff, S. (2022). Single item measures in psychological science. European Journal of Psychological Assessment, 38(1), 1–5. https://doi.org/10.1027/1015-5759/a000699

Berndt, M., Schmidt, F. M., Sailer, M., Fischer, F., Fischer, M. R., & Zottmann, J. M. (2021). Investigating statistical literacy and scientific reasoning & argumentation in medical-, social sciences-, and economics students. Learning and Individual Differences, 86, 101963. https://doi.org/10.1016/j.lindif.2020.101963

Bond, T., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences (4th ed.). Routledge. https://doi.org/10.4324/9780429030499

Chue, K. L. (2022). Using the Rasch model to examine pre-service teachers approach to learning profiles. Methodological Innovations, 15(1), 86–95. https://doi.org/10.1177/20597991221077908

Darman, D. R., Suhandi, A., Kaniawati, I., Samsudin, A., & Wibowo, F. C. (2024). Development and validation of scientific inquiry literacy instrument (SILI) using Rasch measurement model. Education Sciences, 14(3), 322. https://doi.org/10.3390/educsci14030322

Denovan, A., Dagnall, N., Peverell, M., Drinkwater, K. G., Escolà-Gascón, Á., & Carter, D. (2026). Evaluating the psychometric properties of the Generic Conspiracist Beliefs Scale: A Rasch modelling approach. Frontiers in Psychology, 17, 1721668. https://doi.org/10.3389/fpsyg.2026.1721668

Engelhard, G., Jr., & Wang, J. (2024). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences (2nd ed.). Routledge. https://doi.org/10.4324/9781003458746

Forgasz, H., Hall, J., & Robinson, T. (2024). Evaluating pre-service teachers’ statistical literacy capabilities. Mathematics Education Research Journal, 36(1), 231–258. https://doi.org/10.1007/s13394-022-00438-6

Friedrich, A., Schreiter, S., Vogel, M., Becker-Genschow, S., Brünken, R., Kuhn, J., Lehmann, J., & Malone, S. (2024). What shapes statistical and data literacy research in K-12 STEM education? A systematic review of metrics and instructional strategies. International Journal of STEM Education, 11(1), 58. https://doi.org/10.1186/s40594-024-00517-z

Geiger, V., & Schmid, M. (2024). A critical turn in numeracy education and practice. Frontiers in Education, 9, 1363566. https://doi.org/10.3389/feduc.2024.1363566

Hudson, N. W., Anusic, I., Lucas, R. E., & Donnellan, M. B. (2020). Comparing the reliability and validity of global self-report measures of subjective well-being with experiential day reconstruction measures. Assessment, 27(1), 102–116. https://doi.org/10.1177/1073191117744660

Koga, S. (2022). Characteristics of statistical literacy skills from the perspective of critical thinking. Teaching Statistics, 44(2), 59–67. https://doi.org/10.1111/test.12302

Li, J., Pan, D., Zhao, Y., & Long, T. (2026). From video to analytics: A design-based study on fostering critical collective reflection for pre-service science teachers. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10356-y

Liu, Y., Wang, J., Zhang, Z., Wang, J., Luo, T., Lin, S., Li, J., & Xu, S. (2024). Development and validation of an instrument for measuring civic scientific literacy. Disciplinary and Interdisciplinary Science Education Research, 6(1), 6. https://doi.org/10.1186/s43031-023-00092-3

Nima, A. A., Cloninger, K. M., Persson, B. N., Sikström, S., & Garcia, D. (2020). Validation of subjective well-being measures using item response theory. Frontiers in Psychology, 10, 3036. https://doi.org/10.3389/fpsyg.2019.03036

Ozturk, A. B., & Baki, A. (2024). Models of conceptualizing and measuring statistical knowledge for teaching: A critical review. Pedagogical Research, 9(1), em0178. https://doi.org/10.29333/pr/14031

Pretorius, T. B. (2023). The Perceived Stress Scale is essentially unidimensional: Complementary evidence from ancillary bifactor indices and Mokken analysis. Acta Psychologica, 241, 104058. https://doi.org/10.1016/j.actpsy.2023.104058

Ribeiro Santiago, P. H., Nielsen, T., Smithers, L. G., Roberts, R., & Jamieson, L. (2020). Measuring stress in Australia: Validation of the Perceived Stress Scale (PSS-14) in a national sample. Health and Quality of Life Outcomes, 18(1), 100. https://doi.org/10.1186/s12955-020-01343-x

Rizbudiani, A. D., Jaedun, A., Rahim, A., & Nurrahman, A. (2021). Rasch model item response theory (IRT) to analyze the quality of mathematics final semester exam test on system of linear equations in two variables (SLETV). Al-Jabar: Jurnal Pendidikan Matematika, 12(2), 399–412. https://doi.org/10.24042/ajpm.v12i2.9939

Schreiter, S., Friedrich, A., Fuhr, H., Malone, S., Brünken, R., Kuhn, J., & Vogel, M. (2024). Teaching for statistical and data literacy in K-12 STEM education: A systematic review on teacher variables, teacher education, and impacts on classroom practice. ZDM – Mathematics Education, 56(1), 31–45. https://doi.org/10.1007/s11858-023-01531-1

Schüller, K. (2022). Data and AI literacy for everyone. Statistical Journal of the IAOS, 38(2), 477–490. https://doi.org/10.3233/SJI-220941

Shahat, M. A., Boone, W. J., Al-Alawi, K. A., & Al-Balushi, S. M. (2024). Rasch analysis in developing a questionnaire to measure self-efficacy beliefs of Omani preservice science teachers for teaching through engineering design. Humanities and Social Sciences Communications, 11(1), 1528. https://doi.org/10.1057/s41599-024-04087-x

Shaw, A., Kapnek, M., & Morelli, N. A. (2021). Measuring creative self-efficacy: An item response theory analysis of the Creative Self-Efficacy Scale. Frontiers in Psychology, 12, 678033. https://doi.org/10.3389/fpsyg.2021.678033

Shimizu, I., & Ferreira, J. C. (2023). Losing your fear of using R for statistical analysis. Jornal Brasileiro de Pneumologia, 49(3), e20230212. https://doi.org/10.36416/1806-3756/e20230212

Sinclair, S., Hack, T. F., MacInnis, C. C., Jaggi, P., Boss, H., McClement, S., Sinnarajah, A., & Thompson, G. (2021). Development and validation of a patient-reported measure of compassion in healthcare: The Sinclair Compassion Questionnaire (SCQ). BMJ Open, 11(6), e045988. https://doi.org/10.1136/bmjopen-2020-045988

Tesio, L., Caronni, A., Simone, A., Kumbhare, D., & Scarano, S. (2024). Interpreting results from Rasch analysis 2. Advanced model applications and the data-model fit assessment. Disability and Rehabilitation, 46(3), 604–617. https://doi.org/10.1080/09638288.2023.2169772

Umbach, G. (2022). Statistical and data literacy in policy-making. Statistical Journal of the IAOS, 38(2), 445–452. https://doi.org/10.3233/SJI-220962

Welter, V. D. E., Emmerichs-Knapp, L., & Krell, M. (2023). Are we on the way to successfully educating future citizens?—A spotlight on critical thinking skills and beliefs about the nature of science among pre-service biology teachers in Germany. Behavioral Sciences, 13(3), 279. https://doi.org/10.3390/bs13030279

Wu, X., Xu, T., & Zhang, Y. (2023). Research on the data analysis knowledge assessment of pre-service teachers from China based on cognitive diagnostic assessment. Current Psychology, 42(6), 4885–4899. https://doi.org/10.1007/s12144-021-01836-y

Zhunissova, S., Zhussupova, L., Abyzbekova, G., & Islambekuly, B. (2026). Learning to trust experimental data: A validation-based approach to experiment design in pre-service chemistry teacher education. Chemistry Education Research and Practice, 27(3), 1107–1122. https://doi.org/10.1039/D6RP00078A