What is the Relationship between Students’ Computational Thinking Performance and School Achievement?



  • Yerkhan Mindetbay University of Southampton
  • Christian Bokhove University of Southampton
  • John Woollard University of Southampton


Computational Thinking, student achievement, testing, predictors, multiple-choice questions, item response theory


This study investigates the relationship between computational thinking performance and general school achievement and explores to see if computational thinking performance can be predicted by algebra and informatics achievement. The sample group of 775 grade 8 students was drawn from 28 secondary schools across Kazakhstan. The students responded to a Computational Thinking Performance test of 50 multiple-choice questions and Computational Thinking Scale questionnaire. The test covers the concepts: logical thinking, generalisation and abstraction. The validity and reliability of the multiple-choice questions are tested using the Item Response Theory. The Likert type questionnaire covers five factors: creativity, algorithmic thinking, cooperation, critical thinking and problem solving. School achievement results (secondary data) include scores for a number of school subjects. The results of the study showed that the multiple-choice questions are valid and a reliable tool to measure computational thinking performance of students. Algebra, general school achievement and students’ perception of their computational thinking skills were significant predictors of computational thinking performance. The results revealed no gender difference in computational thinking performance and perceptions of computational thinking. The findings regarding the relationship between computational thinking performance, the students’ general school achievement and perceptions of computational thinking skills are compared and discussed.


Download data is not yet available.


Ambrosio, A. P., Almeida, L. da S., Franco, A., & Macedo, J. (2014). Exploring Core Cognitive Skills of Computational Thinking. In PPIG 2014 - 25th Annual Workshop. Sussex: University of Sussex.

Barefoot. (2014). Computational thinking. Crown copyright. Retrieved April 13, 2017 from http://barefootcas.org.uk/barefoot-primary-computing-resources/concepts/computational-thinking/

Becker, W. E., & Johnston, C. G. (1999). The Relationship between Multiple Choice and Essay Response Questions in Assessing Economics Understanding. Economic Record, 75(231), 348–357. doi:10.1111/j.1475-4932.1999.tb02571.x DOI: https://doi.org/10.1111/j.1475-4932.1999.tb02571.x

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. American Educational Research Association. Vancouver. doi:

Chuang, H. C., Hu, C. F., Wu, C. C., & Lin, Y. T. (2015). Computational thinking curriculum for K-12 education - A Delphi survey. In Proceedings - 2015 International Conference on Learning and Teaching in Computing and Engineering, LaTiCE 2015 (pp. 213–214). Taipei. doi:10.1109/LaTiCE.2015.44 DOI: https://doi.org/10.1109/LaTiCE.2015.44

CS Unplugged. (2016). What is Computational Thinking. Computational Thinking and CS Unplugged. Retrieved January 29, 2018, from https://cs-unplugged.appspot.com/en/computational-thinking/

Csizmadia, A., Curzon, P., Dorling, M., Humphreys, S., Ng, T., Selby, C., & Woollard, J. (2015). Computational thinking A guide for teachers. Computing At School.

Denning, P. J. (2009). The profession of IT Beyond computational thinking. Communications of the ACM, 52(6), 28. doi:10.1145/1516046.1516054 DOI: https://doi.org/10.1145/1516046.1516054

Denning, P. J. (2017). Remaining trouble spots with computational thinking. Communications of the ACM, 60(6), 33–39. doi:10.1145/2998438 DOI: https://doi.org/10.1145/2998438

Downing, S. M., & Haladyna, T. M. (2006). Handbook of test development (1st ed.). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Dufresne, R. J., Leonard, W. J., & Gerace, W. J. (2002). Marking sense of students’ answers to multiple-choice questions. The Physics Teacher, 40(3), 174–180. doi:10.1119/1.1466554 DOI: https://doi.org/10.1119/1.1466554

Durak, H. Y., & Saritepeci, M. (2017). Analysis of the relation between computational thinking skills and various variables with the structural equation model. Computers & Education, 116, 191–202. doi:10.1016/j.compedu.2017.09.004 DOI: https://doi.org/10.1016/j.compedu.2017.09.004

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. SAGE Publications Ltd (3rd ed.). London.

Frey, B. B., Petersen, S., Edwards, L. M., Pedrotti, J. T., & Peyton, V. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357–364. doi:10.1016/j.tate.2005.01.008 DOI: https://doi.org/10.1016/j.tate.2005.01.008

Gayef, A., Oner, C., & Telatar, B. (2014). Is asking same question in different ways has any impact on student achievement? Procedia - Social and Behavioral Sciences, 152(212), 339–342. doi:10.1016/j.sbspro.2014.09.206 DOI: https://doi.org/10.1016/j.sbspro.2014.09.206

Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review. Review of Educational Research, 86(6), 1082–1116. doi:10.3102/0034654317726529 DOI: https://doi.org/10.3102/0034654317726529

Google for Education. (2015). Exploring Computational Thinking. Retrieved April 3, 2017, from https://edu.google.com/resources/programs/exploring-computational-thinking/#!home

Gouws, L., Bradshaw, K., & Wentworth, P. (2013). First year student performance in a test for computational thinking. ACM International Conference Proceeding Series, 271 – 277. doi:10.1145/2513456.2513484 DOI: https://doi.org/10.1145/2513456.2513484

Grover, S. (2011). Robotics and Engineering for Middle and High School Students to Develop Computational Thinking Robotics and Engineering for Middle and High School Students to Develop Computational Thinking, (650), 1–15.

Grover, S. (2015). “Systems of Assessments” for Deeper Learning of Computational Thinking in K-12. Annual Meeting of the American Educational Research Association, (650).

Grover, S., & Pea, R. (2013). Computational Thinking in K–12: A Review of the State of the Field. Educational Researcher, 42(1), 38–43. doi:10.3102/0013189X12463051 DOI: https://doi.org/10.3102/0013189X12463051

Guzdial, M. (2015). Learner-Centered Design of Computing Education: Research on Computing for Everyone. Synthesis Lectures on Human-Centered Informatics, 8(6), 1–165. doi:10.2200/S00684ED1V01Y201511HCI033 DOI: https://doi.org/10.2200/S00684ED1V01Y201511HCI033

Hancock, G. R. (1994). Cognitive complexity and the comparability of multiple-choice and constructed-response test formats. Journal of Experimental Education, 62(2), 143–157. doi:10.1080/00220973.1994.9943836 DOI: https://doi.org/10.1080/00220973.1994.9943836

International Society for Technology in Education (ISTE), & Computer Science Teachers Association (CSTA). (2011). Operational Definition of Computational Thinking.

Kallia, M. (2017). Assessment in Computer Science courses : A Literature Review. London.

Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. doi:10.1016/j.chb.2017.01.005 DOI: https://doi.org/10.1016/j.chb.2017.01.005

Korkmaz, Ö., Çak?r, R., Özden, M. Y., Oluk, A., & Sar?o?lu, S. (2015). Bireylerin Bilgisayarca Dü?ünme Becerilerinin Farkl? De?i?kenler Aç?s?ndan ?ncelenmesi. Ondokuz May?s Üniversitesi E?itim Fakültesi Dergisi, 34(2), 68–87. doi:10.7822/omuefd.34.2.5

Korkmaz, Ö., Çak?r, R., & Özden, Y. (2015). Computational Thinking Levels Scale (CTLS) Adaptation for Secondary School Level. Gazi Journal of Education Sciences, 143–162.

Korucu, A. T., Gencturk, A. T., & Gundogdu, M. M. (2017). Examination of the Computational Thinking Skills of Students. Journal of Learning and Teaching in Digital Age, 2(1), 11–19.

Martinez, M. E. (1999). Cognition and the question of test item format. Educational Psychologist, 34(4), 207–218. doi:10.1207/s15326985ep3404_2 DOI: https://doi.org/10.1207/s15326985ep3404_2

Moreno-Leon, J., Robles, G., & Román-González, M. (2016). Comparing Computational Thinking Development Assessment Scores with Software Complexity Metrics. In 2016 IEEE Global Engineering Education Conference (EDUCON) (pp. 1040–1045). Abu Dhabi. doi:10.1109/EDUCON.2016.7474681 DOI: https://doi.org/10.1109/EDUCON.2016.7474681

National Academy of Education. (2016). Updated curriculum by the National Academy of Education. Astana: National Academy of Education named after Y.Altynsarin. Retrieved December 12, 2016, from http://nao.kz/loader/load/260

Oluk, A., & Korkmaz, Ö. (2016). Comparing Students’ Scratch Skills with Their Computational Thinking Skills in Terms of Different Variables. International Journal of Modern Education and Computer Science, 8(11), 1–7. doi:10.5815/ijmecs.2016.11.01 DOI: https://doi.org/10.5815/ijmecs.2016.11.01

Papert, S. (1980). Mindstorm. Journal of Chemical Information and Modeling. doi:10.1017/CBO9781107415324.004 DOI: https://doi.org/10.1017/CBO9781107415324.004

Paxton, M. (2000). A linguistic perspective on multiple choice questioning. Assessment and Evaluation in Higher Education, 25(2), 109–119. doi:10.1080/713611429 DOI: https://doi.org/10.1080/713611429

Reynolds, C. R., Livingston, R. B., & Willson, V. (2009). Measurement and assessment in education (2nd ed.). New Jersey: Pearson.

Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2016). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior. doi:10.1016/j.chb.2016.08.047 DOI: https://doi.org/10.1016/j.chb.2016.08.047

Selby, C. (2014). How can the teaching of programming be used to enhance computational thinking skills? University of Southampton.

Shaniyev, Y., Gesen, I., Aidarbayev, N., Akhmetov, N., & Yerzhanov, E. (2017). Informatics - A bilingual textbook - Grade 8 (1st ed.). Astana: Astana Kitap.

Simkin, M. G., & Kuechler, W. L. (2005). Multiple-Choice Tests and Student Understanding: What Is the Connection? Decision Sciences Journal of Innovative Education, 3(1), 73–98. doi:10.1111/j.1540-4609.2005.00053.x DOI: https://doi.org/10.1111/j.1540-4609.2005.00053.x

Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining Computational Thinking for Mathematics and Science Classrooms. Journal of Science Education and Technology, 25(1), 127–147. doi:10.1007/s10956-015-9581-5 DOI: https://doi.org/10.1007/s10956-015-9581-5

Werner, L., Denner, J., & Campe, S. (2015). Children Programming Games: A Strategy for Measuring Computational Learning. Trans. Comput. Educ., 14(4), 24:1–24:22. doi:10.1145/2677091 DOI: https://doi.org/10.1145/2677091

Werner, L., Denner, J., Campe, S., & Kawamoto, D. C. (2012). The Fairy Performance Assessment : Measuring Computational Thinking in Middle School. Proceedings of the 43rd ACM Technical Symposium on Computer Science Education - SIGCSE ’12, 215–220. doi:10.1145/2157136.2157200 DOI: https://doi.org/10.1145/2157136.2157200

Wing, J. (2011). Research notebook: Computational thinking—What and why? The Link Magazine. DOI: https://doi.org/10.1109/VLHCC.2011.6070404



How to Cite

Mindetbay, Y., Bokhove, C., & Woollard, J. (2019). What is the Relationship between Students’ Computational Thinking Performance and School Achievement?. International Journal of Computer Science Education in Schools, 2(5), 3–19. https://doi.org/10.21585/ijcses.v0i0.45