Semantic Analyses of Open-Ended Responses From Professional Development Workshop Promoting Computational Thinking in Rural Schools


  • Amber Gillenwaters Missouri State University
  • Razib Iqbal Missouri State University
  • Diana Piccolo Missouri State University
  • Tammi Davis Missouri State University
  • Keri Franklin Missouri State University
  • David Cornelison
  • Judith Martinez Missouri State University
  • Andrew Homburg Missouri State University
  • Julia Cottrell Missouri State University
  • Melissa Page The Evaluation Group


K-12 education, rural classroom, formative assessment, teaching strategy, semantic analysis, thematic analysis, sentiment analysis


The development of curriculum and access to educational resources related to applied computing is lacking for students in K-12 schools particularly in rural areas, despite the large and growing demand for computing skills in the job market. Motivated by this need, an interdisciplinary professional development workshop was designed to promote computational thinking and curriculum integration among teachers involved in teaching core curricula including writing, math, science, and social studies in grades 3-8 in a rural midwestern state in the USA, as part of a longitudinal grant-funded program. Open-text feedback was collected before, during, and immediately after the workshop in response to multiple types of formative assessments. In this paper, we present several forms of data representation from exploratory textual analyses based on the feedback collected from the workshop participants. Semantic analysis tools including sentiment analysis and thematic analysis facilitated the identification of common themes in perception among grade 3-8 teachers relating to the implementation of computational concepts in their classrooms. Results suggest that these techniques can be useful in evaluating open-ended feedback to represent patterns of response which may aid in the identification of actionable insights related to adult learner perceptions, including interest and self-efficacy.


Download data is not yet available.


Benotti, L., Martinez, M. C., & Schapachnik, F. (2017). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179-192. Retrieved from DOI:

Bird, S. (2006, July). NLTK: The natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions (pp. 69-72). Retrieved from DOI:

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5–31. Retrieved from DOI:

Brown, E., & Brown, R. (2020). The Effect of Advanced Placement Computer Science Course Taking on College Enrollment. West Coast Analytics. Retrieved from

Chamblee, G., & Slough, S. (2004). Using the concerns-based adoption model to assess changes in technology implementation. In Society for Information Technology & Teacher Education International Conference (pp. 864-871). Association for the Advancement of Computing in Education (AACE). Retrieved from

Chen, C. M., Li, M. C., & Huang, Y. L. (2020). Developing an instant semantic analysis and feedback system to facilitate learning performance of online discussion. Interactive Learning Environments, 1-19. Retrieved from

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–79. Retrieved from DOI:

Clarke, V., & Braun, V. (2014). Thematic analysis. In Encyclopedia of Critical Psychology (pp. 1947–1952). Springer, New York, NY. Retrieved from DOI:

Code Advocacy Coalition (2021). State of computer science education: Accelerating action through advocacy. Retrieved from

Djudin, T. (2021). Promoting students’ conceptual change by integrating the 3-2-1 reading technique with refutation text in the physics learning of buoyancy. Journal of Turkish Science Education, 18(2), 290–303. Retrieved from

Education Commission of the States (2019). Stem content: Give more united states students early access to computer science. Retrieved from

Feldman, R. (2013). Techniques and applications for sentiment analysis. Communications of the ACM, 56(4), 82-89. Retrieved from DOI:

Gabby, S., Avargil, S., Herscovitz, O., & Dori, Y. J. (2017). The case of middle and high school chemistry teachers implementing technology: Using the concerns-based adoption model to assess change processes. Chemistry Education Research and Practice, 18(1), 214-232. Retrieved from DOI:

Gottipati, S., Shankararaman, V., & Lin, J. R. (2018). Text analytics approach to extract course improvement suggestions from students’ feedback. Research and Practice in Technology Enhanced Learning, 13(1), 1-19. Retrieved from DOI:

Graham, S., Bruch, J., Fitzgerald, J., Friedrich, L. D., Furgeson, J., Greene, K., ... & Smither Wulsin, C. (2016). Teaching Secondary Students to Write Effectively. Educator's Practice Guide. What Works Clearinghouse.™ NCEE 2017-4002. What Works Clearinghouse.

Guskey, T. R. (2005). Formative Classroom Assessment and Benjamin S. Bloom: Theory, Research, and Implications [Paper presentation]. Annual Meeting of the American Educational Research Association, Montreal, Canada. Retrieved from

Guzdial, M., & Hill, R. K. (2019). Getting high school, college students interested in CS. Communications of the ACM, 62(12), 10–11. Retrieved from

Hamilton, W. L., Clark, K., Leskovec, J., & Jurafsky, D. (2016, November). Inducing domain-specific sentiment lexicons from unlabeled corpora. In Proceedings of the Conference on Empirical methods in Natural Language Processing. NIH Public Access. Retrieved from DOI:

Hashemi, A., Mobini, F., & Karimkhanlooie, G. (2016). The impact of content-based pre-reading activities on iranian high school EFL learners’ reading comprehension. Journal of Language Teaching and Research, 7(1), 137. Retrieved from DOI:

Helminski, L. (1995). Total quality in instruction: A systems approach. In H. V. Roberts (Ed.), Academic initiatives in total quality for higher education (pp. 309–362). Milwaukee, WI: ASQC Quality Press.

Hu, M., & Liu, B. (2004, August). Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 168-177). Retrieved from DOI:

Hubwieser, P., Giannakos, M. N., Berges, M., Brinda, T., Diethelm, I., Magenheim, J., ... & Jasute, E. (2015). A global snapshot of computer science education in K-12 schools. In Proceedings of the 2015 ITiCSE on Working Group Reports (pp. 65-83). Retrieved from DOI:

Lamb, R., Annetta, L., Vallett, D., Firestone, J., Schmitter-Edgecombe, M., Walker, H., ... Hoston, D. (2018). Psychosocial factors impacting STEM career selection. The Journal of Educational Research, 111(4), 446–458. Retrieved from DOI:

Liu, J., Lin, C. H., Hasson, E. P., & Barnett, Z. D. (2011, March). Introducing computer science to K-12 through a summer computing workshop for teachers. In Proceedings of the 42nd ACM technical symposium on computer science education (pp. 389-394). Retrieved from DOI:

Loughran, T., & McDonald, B. (2011). When is a liability not a liability? Textual analysis, dictionaries, and 10‐Ks. The Journal of Finance, 66(1), 35-65. Retrieved from DOI:

Mahadeo, J., Hazari, Z., & Potvin, G. (2020). Developing a computing identity framework: Understanding computer science and information technology career choice. ACM Transactions on Computing Education (TOCE), 20(1), 1–14. Retrieved from

Marré, A. (2017). Rural education at a glance, 2017 edition. U.S. Department of Agriculture, Economic Research Service, Economic Information Bulletin, 171. Retrieved from

Masood, K., Khan, M. A., Saeed, U., Al Ghamdi, M. A., Asif, M., & Arfan, M. (2022). Semantic Analysis to Identify Students’ Feedback. The Computer Journal, 65(4), 918-925. Retrieved from

Maxwell, J. A., & Chmiel, M. (2014). Notes toward a theory of qualitative data analysis. The SAGE Handbook of Qualitative Data Analysis, 21-34. DOI:

Mohammad, S. M., & Turney, P. D. (2013). Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29(3), 436-465. Retrieved from DOI:

Newlove, B. W., & Hall, G. E. (1976). A Manual for Assessing Open-Ended Statements of Concern About an Innovation. Retrieved from

Nielsen, F. Å. (2011, March). AFINN: A new word list for sentiment analysis on Twitter. Informatics and Mathematical Modelling, Technical University of Denmark. Retrieved from

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1). Retrieved from DOI:

Ogle, D. M. (1986). K-W-L: A Teaching Model That Develops Active Reading of Expository Text. The ReadingTeacher, 39(6), 564–570. DOI:

Patka, M., Wallin-Ruschman, J., Wallace, T., & Robbins, C. (2016). Exit cards: creating a dialogue for continuous evaluation. Teaching in Higher Education, 21(6), 659–668. Retrieved from DOI:

Salehi, S., Wang, K. D., Toorawa, R., & Wieman, C. (2020, February). Can majoring in computer science improve general problem-solving skills? In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 156–161). Retrieved from

Showalter, D., Klein, R., Johnson, J., & Hartman, S. L. (2017). Why Rural Matters 2015-2016: Understanding the Changing Landscape. A Report of the Rural School and Community Trust. Rural School and Community Trust. Retrieved from

Steele, J., & Dyer, T. (2014). Use of KWLS in the online classroom as it correlates to increased participation. Journal of Instructional Research, 3, 8–14. Retrieved from

U.S. Bureau of Labor Statistics (2021, May). Occupational Outlook Handbook: Computer and information technology occupations. Retrieved from

Wang, J., & Hejazi Moghadam, S. (2017, March). Diversity barriers in K-12 computer science education: Structural and social. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (pp. 615-620). Retrieved from DOI:



How to Cite

Gillenwaters, A., Iqbal, R., Piccolo, D., Davis, T., Franklin, K., Cornelison, D., Martinez, J., Homburg, A., Cottrell, J., & Page, M. (2023). Semantic Analyses of Open-Ended Responses From Professional Development Workshop Promoting Computational Thinking in Rural Schools. International Journal of Computer Science Education in Schools, 6(1), 59–78.