Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons
2024
Преузимање 🢃
Конференцијски прилог (Објављена верзија)
Метаподаци
Приказ свих података о документуАпстракт
Contribution
The European key competences framework distinguishes digital literacy among the eight key competences in education (EU, 2006), which is why policy makers and practitioners in Europe and beyond put a great effort in introducing changes that will support the development of these skills (Punie et al, 2017). Digital literacy consists of “knowledge, skills, values and awareness that are required when using ICT and digital media to perform tasks, solve problems, communicate, manage information, collaborate, create and share content, build knowledge effectively, efficiently, appropriately, critically, creatively, autonomously, flexibly, ethically, reflectively for work, leisure, participation, learning, socializing, consuming, and empowerment” (Ferrari, 2012). This competence is required for a full participation in the contemporary society, and it is getting more and more important as requests for using digital resources are expending rapidly in many jobs and other activities, w...hich was especially notable during the pandemic (Kovács Cerović et al, 2021).
Recent Eurostat (2021) data show that about 50% of individuals in the European area have basic overall digital skills, ranging from 25% in Albania to 80% in Scandinavian countries. The latest IEA International Computer and Information Literacy report showed that 1 in 5 students worldwide did not have a functional working knowledge of computers at the end of lower secondary education, with wider differences observed within countries than between countries (Fraillon et al., 2021).
The urging need to ensure a stimulative learning environment for supporting the development of students’ digital literacy prompts teachers to come up with innovative teaching solutions. One such solution comes from a general upper secondary school in Serbia and was awarded with the European Training Foundation Innovative teaching and learning award in 2022 under the “Creating New learning” initiative. A Computer science teacher together with her students have created a game-based learning resource, called “The Escape room”, that corresponds to lower secondary Computer Science curriculum. The resource covers the seven topics of digital literacy: 1. Search the Internet; 2. Reliability of information on the internet and copyright, 3. Online identity; 4. Safety on the Internet; 5. Safe use of digital devices; 6. E-mail and working with shared documents; 7. Open data. The game is set on the Minecraft platform and uses the principles of the escape room game where a player goes through the room, explores a topic by reading information points, and has to answer questions to exit the room and move to the next one.
Previous studies dealing with game-based teaching resources haven’t provided conclusive evidence on whether they are beneficial to learning. A thorough review suggests that game-based resources support motivation for learning (Divjak & Tomić, 2011), however a recent meta-analysis revealed varying findings on their impacts on learning gains, with both positive and negative effects observed, and outcomes being influenced by factors like ease of use or cognitive workload (Zhonggen, 2021). In addition, a study investigating the impact of the experience of enjoyment while playing a game on learning gains found no connection between the two (Iten & Petko, 2016).
The current study was supported by the European Training Foundation initiative “Creating New Learning” and looked into the effects of the Escape room practice. The study took place in two elementary schools and involved three Computer science teachers – the one who developed the practice and two teachers who tried out the practices with their students, as well as researchers who led the study. Considering the ambiguity of previous findings, the study had two goals: 1. to assess whether the Escape room practice improves the motivation for learning, and 2. to assess whether the Escape room practice contributes to learning gains.
Method
The study used mix methods and was participatory (Bergold & Thomas, 2012), meaning that all decisions were made jointly by the teachers and the researchers, and that the teachers participated in interpretation of findings. The study relied on a quasi-experimental design (Todorović, 2008) which enabled a reliable assessment of the practice effects through comparison of experimental and control group results. The participating students were in grades 5 to 8. At the beginning of the study, all students undertook a digital literacy test. To make the experimental and control group similar in terms of their initial digital literacy, allocation of the classes was based on the classes’ average digital literacy scores. In each of the four grades, half of the (whole) classes were assigned to control group and other half to experimental group. Across the four grades, there were 18 classes in the experimental group with a total of 217 students, and 18 control group classes with a total of 201 students. Over the course of five weeks, the experimental group classes had their Computer science lessons conducted with the Escape room, while the control group classes had their lessons the usual way which included frontal teaching, discussions, students’ presentation, and problem-based learning - depending on the grade and a lesson. Data was collected from 360 students whose parents gave consent. To assess effects on motivation for learning, after each lesson students filled out a short questionnaire assessing their intrinsic motivation. The questionnaire had seven items (e.g., “I think this activity was quite enjoyable”) followed by a 10-point scale and was based on the Self-determination theory (Ryan & Deci, 2020). This data was analysed using paired-samples t-test. Group effects were estimated on both school level and the whole sample level, thus checking for the moderating effect of a teacher. To assess the effects of the practice on learning outcomes, after the five weeks all students undertook another digital literacy test. Data from this instrument were analysed by using repeated measures analysis of variance and inspecting time X group interactions. The moderating effect of the teacher was also investigated. To make interpretation of quantitative results more reliable and to gather additional insight, the study also included a qualitative method. Additional data were collected through interviews with the two teachers and two focus group discussions with students. These data were analysed on the basis of the thematic analysis (Braun & Clarke, 2012).
Expected Outcomes
The results showed that the practices contributed to boosting motivation, while it didn’t have impact on learning. However, the practice’s effects on the intrinsic motivation measure were moderated by the grade and the teacher, indicating the importance of contextual factors in the implementation of the practice. In school A, students from the experimental group from grades 6 to 8 reported higher motivation than the control group students (p<.05), while grade 5 students from the control group were more motivated than their experimental group counterparts (p<.05). Being that five graders from this school had the lowest initial digital literacy scores, this finding suggest that a certain starting level of digital literacy is necessary for the practice to be effects. Contrary, the use of the practice by insufficiently skilled students could have negative effects, probably by affecting their perceived competence during the learning activity. School B, which had technical obstacles that caused interruptions and prevented an autonomous use of the game by students, had mixed results. The practice had impact on motivation in grades 5 and 8 (p<.05), while no difference was found in grades 6 and 7 (p>.05). The practice didn’t show effects on learning outcomes measured by the digital literacy test in school A (p between .159 and .922). However, in school B, where students experienced technical difficulties while using the game, the control group had better achievement on the posttest measure than their experimental group counterparts (p<.05). This finding stressed the importance of ensuring the proper technical conditions prior to implementing the practice. Students’ and teachers’ insights revealed enablers and barriers to the practice implementation and supported fine nuancing of the quantitative findings, thus enlightening the mechanisms through which the practice impacted learning, which is applicable to other ICT based teaching resources as well.
Кључне речи:
digital literacy / games / learning / computer science / education / minecraftИзвор:
European Conference on Educational Research 2024, Nicosia, Cyprus, 2024URI
https://eera-ecer.de/ecer-programmes/conference/29/contribution/59177http://reff.f.bg.ac.rs/handle/123456789/6639
Институција/група
Psihologija / PsychologyTY - CONF AU - Mićić, Katarina AU - Veljković, Katarina AU - Stančić, Milan AU - van Uden, Jolien PY - 2024 UR - https://eera-ecer.de/ecer-programmes/conference/29/contribution/59177 UR - http://reff.f.bg.ac.rs/handle/123456789/6639 AB - Contribution The European key competences framework distinguishes digital literacy among the eight key competences in education (EU, 2006), which is why policy makers and practitioners in Europe and beyond put a great effort in introducing changes that will support the development of these skills (Punie et al, 2017). Digital literacy consists of “knowledge, skills, values and awareness that are required when using ICT and digital media to perform tasks, solve problems, communicate, manage information, collaborate, create and share content, build knowledge effectively, efficiently, appropriately, critically, creatively, autonomously, flexibly, ethically, reflectively for work, leisure, participation, learning, socializing, consuming, and empowerment” (Ferrari, 2012). This competence is required for a full participation in the contemporary society, and it is getting more and more important as requests for using digital resources are expending rapidly in many jobs and other activities, which was especially notable during the pandemic (Kovács Cerović et al, 2021). Recent Eurostat (2021) data show that about 50% of individuals in the European area have basic overall digital skills, ranging from 25% in Albania to 80% in Scandinavian countries. The latest IEA International Computer and Information Literacy report showed that 1 in 5 students worldwide did not have a functional working knowledge of computers at the end of lower secondary education, with wider differences observed within countries than between countries (Fraillon et al., 2021). The urging need to ensure a stimulative learning environment for supporting the development of students’ digital literacy prompts teachers to come up with innovative teaching solutions. One such solution comes from a general upper secondary school in Serbia and was awarded with the European Training Foundation Innovative teaching and learning award in 2022 under the “Creating New learning” initiative. A Computer science teacher together with her students have created a game-based learning resource, called “The Escape room”, that corresponds to lower secondary Computer Science curriculum. The resource covers the seven topics of digital literacy: 1. Search the Internet; 2. Reliability of information on the internet and copyright, 3. Online identity; 4. Safety on the Internet; 5. Safe use of digital devices; 6. E-mail and working with shared documents; 7. Open data. The game is set on the Minecraft platform and uses the principles of the escape room game where a player goes through the room, explores a topic by reading information points, and has to answer questions to exit the room and move to the next one. Previous studies dealing with game-based teaching resources haven’t provided conclusive evidence on whether they are beneficial to learning. A thorough review suggests that game-based resources support motivation for learning (Divjak & Tomić, 2011), however a recent meta-analysis revealed varying findings on their impacts on learning gains, with both positive and negative effects observed, and outcomes being influenced by factors like ease of use or cognitive workload (Zhonggen, 2021). In addition, a study investigating the impact of the experience of enjoyment while playing a game on learning gains found no connection between the two (Iten & Petko, 2016). The current study was supported by the European Training Foundation initiative “Creating New Learning” and looked into the effects of the Escape room practice. The study took place in two elementary schools and involved three Computer science teachers – the one who developed the practice and two teachers who tried out the practices with their students, as well as researchers who led the study. Considering the ambiguity of previous findings, the study had two goals: 1. to assess whether the Escape room practice improves the motivation for learning, and 2. to assess whether the Escape room practice contributes to learning gains. Method The study used mix methods and was participatory (Bergold & Thomas, 2012), meaning that all decisions were made jointly by the teachers and the researchers, and that the teachers participated in interpretation of findings. The study relied on a quasi-experimental design (Todorović, 2008) which enabled a reliable assessment of the practice effects through comparison of experimental and control group results. The participating students were in grades 5 to 8. At the beginning of the study, all students undertook a digital literacy test. To make the experimental and control group similar in terms of their initial digital literacy, allocation of the classes was based on the classes’ average digital literacy scores. In each of the four grades, half of the (whole) classes were assigned to control group and other half to experimental group. Across the four grades, there were 18 classes in the experimental group with a total of 217 students, and 18 control group classes with a total of 201 students. Over the course of five weeks, the experimental group classes had their Computer science lessons conducted with the Escape room, while the control group classes had their lessons the usual way which included frontal teaching, discussions, students’ presentation, and problem-based learning - depending on the grade and a lesson. Data was collected from 360 students whose parents gave consent. To assess effects on motivation for learning, after each lesson students filled out a short questionnaire assessing their intrinsic motivation. The questionnaire had seven items (e.g., “I think this activity was quite enjoyable”) followed by a 10-point scale and was based on the Self-determination theory (Ryan & Deci, 2020). This data was analysed using paired-samples t-test. Group effects were estimated on both school level and the whole sample level, thus checking for the moderating effect of a teacher. To assess the effects of the practice on learning outcomes, after the five weeks all students undertook another digital literacy test. Data from this instrument were analysed by using repeated measures analysis of variance and inspecting time X group interactions. The moderating effect of the teacher was also investigated. To make interpretation of quantitative results more reliable and to gather additional insight, the study also included a qualitative method. Additional data were collected through interviews with the two teachers and two focus group discussions with students. These data were analysed on the basis of the thematic analysis (Braun & Clarke, 2012). Expected Outcomes The results showed that the practices contributed to boosting motivation, while it didn’t have impact on learning. However, the practice’s effects on the intrinsic motivation measure were moderated by the grade and the teacher, indicating the importance of contextual factors in the implementation of the practice. In school A, students from the experimental group from grades 6 to 8 reported higher motivation than the control group students (p<.05), while grade 5 students from the control group were more motivated than their experimental group counterparts (p<.05). Being that five graders from this school had the lowest initial digital literacy scores, this finding suggest that a certain starting level of digital literacy is necessary for the practice to be effects. Contrary, the use of the practice by insufficiently skilled students could have negative effects, probably by affecting their perceived competence during the learning activity. School B, which had technical obstacles that caused interruptions and prevented an autonomous use of the game by students, had mixed results. The practice had impact on motivation in grades 5 and 8 (p<.05), while no difference was found in grades 6 and 7 (p>.05). The practice didn’t show effects on learning outcomes measured by the digital literacy test in school A (p between .159 and .922). However, in school B, where students experienced technical difficulties while using the game, the control group had better achievement on the posttest measure than their experimental group counterparts (p<.05). This finding stressed the importance of ensuring the proper technical conditions prior to implementing the practice. Students’ and teachers’ insights revealed enablers and barriers to the practice implementation and supported fine nuancing of the quantitative findings, thus enlightening the mechanisms through which the practice impacted learning, which is applicable to other ICT based teaching resources as well. C3 - European Conference on Educational Research 2024, Nicosia, Cyprus T1 - Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons UR - https://hdl.handle.net/21.15107/rcub_reff_6639 ER -
@conference{ author = "Mićić, Katarina and Veljković, Katarina and Stančić, Milan and van Uden, Jolien", year = "2024", abstract = "Contribution The European key competences framework distinguishes digital literacy among the eight key competences in education (EU, 2006), which is why policy makers and practitioners in Europe and beyond put a great effort in introducing changes that will support the development of these skills (Punie et al, 2017). Digital literacy consists of “knowledge, skills, values and awareness that are required when using ICT and digital media to perform tasks, solve problems, communicate, manage information, collaborate, create and share content, build knowledge effectively, efficiently, appropriately, critically, creatively, autonomously, flexibly, ethically, reflectively for work, leisure, participation, learning, socializing, consuming, and empowerment” (Ferrari, 2012). This competence is required for a full participation in the contemporary society, and it is getting more and more important as requests for using digital resources are expending rapidly in many jobs and other activities, which was especially notable during the pandemic (Kovács Cerović et al, 2021). Recent Eurostat (2021) data show that about 50% of individuals in the European area have basic overall digital skills, ranging from 25% in Albania to 80% in Scandinavian countries. The latest IEA International Computer and Information Literacy report showed that 1 in 5 students worldwide did not have a functional working knowledge of computers at the end of lower secondary education, with wider differences observed within countries than between countries (Fraillon et al., 2021). The urging need to ensure a stimulative learning environment for supporting the development of students’ digital literacy prompts teachers to come up with innovative teaching solutions. One such solution comes from a general upper secondary school in Serbia and was awarded with the European Training Foundation Innovative teaching and learning award in 2022 under the “Creating New learning” initiative. A Computer science teacher together with her students have created a game-based learning resource, called “The Escape room”, that corresponds to lower secondary Computer Science curriculum. The resource covers the seven topics of digital literacy: 1. Search the Internet; 2. Reliability of information on the internet and copyright, 3. Online identity; 4. Safety on the Internet; 5. Safe use of digital devices; 6. E-mail and working with shared documents; 7. Open data. The game is set on the Minecraft platform and uses the principles of the escape room game where a player goes through the room, explores a topic by reading information points, and has to answer questions to exit the room and move to the next one. Previous studies dealing with game-based teaching resources haven’t provided conclusive evidence on whether they are beneficial to learning. A thorough review suggests that game-based resources support motivation for learning (Divjak & Tomić, 2011), however a recent meta-analysis revealed varying findings on their impacts on learning gains, with both positive and negative effects observed, and outcomes being influenced by factors like ease of use or cognitive workload (Zhonggen, 2021). In addition, a study investigating the impact of the experience of enjoyment while playing a game on learning gains found no connection between the two (Iten & Petko, 2016). The current study was supported by the European Training Foundation initiative “Creating New Learning” and looked into the effects of the Escape room practice. The study took place in two elementary schools and involved three Computer science teachers – the one who developed the practice and two teachers who tried out the practices with their students, as well as researchers who led the study. Considering the ambiguity of previous findings, the study had two goals: 1. to assess whether the Escape room practice improves the motivation for learning, and 2. to assess whether the Escape room practice contributes to learning gains. Method The study used mix methods and was participatory (Bergold & Thomas, 2012), meaning that all decisions were made jointly by the teachers and the researchers, and that the teachers participated in interpretation of findings. The study relied on a quasi-experimental design (Todorović, 2008) which enabled a reliable assessment of the practice effects through comparison of experimental and control group results. The participating students were in grades 5 to 8. At the beginning of the study, all students undertook a digital literacy test. To make the experimental and control group similar in terms of their initial digital literacy, allocation of the classes was based on the classes’ average digital literacy scores. In each of the four grades, half of the (whole) classes were assigned to control group and other half to experimental group. Across the four grades, there were 18 classes in the experimental group with a total of 217 students, and 18 control group classes with a total of 201 students. Over the course of five weeks, the experimental group classes had their Computer science lessons conducted with the Escape room, while the control group classes had their lessons the usual way which included frontal teaching, discussions, students’ presentation, and problem-based learning - depending on the grade and a lesson. Data was collected from 360 students whose parents gave consent. To assess effects on motivation for learning, after each lesson students filled out a short questionnaire assessing their intrinsic motivation. The questionnaire had seven items (e.g., “I think this activity was quite enjoyable”) followed by a 10-point scale and was based on the Self-determination theory (Ryan & Deci, 2020). This data was analysed using paired-samples t-test. Group effects were estimated on both school level and the whole sample level, thus checking for the moderating effect of a teacher. To assess the effects of the practice on learning outcomes, after the five weeks all students undertook another digital literacy test. Data from this instrument were analysed by using repeated measures analysis of variance and inspecting time X group interactions. The moderating effect of the teacher was also investigated. To make interpretation of quantitative results more reliable and to gather additional insight, the study also included a qualitative method. Additional data were collected through interviews with the two teachers and two focus group discussions with students. These data were analysed on the basis of the thematic analysis (Braun & Clarke, 2012). Expected Outcomes The results showed that the practices contributed to boosting motivation, while it didn’t have impact on learning. However, the practice’s effects on the intrinsic motivation measure were moderated by the grade and the teacher, indicating the importance of contextual factors in the implementation of the practice. In school A, students from the experimental group from grades 6 to 8 reported higher motivation than the control group students (p<.05), while grade 5 students from the control group were more motivated than their experimental group counterparts (p<.05). Being that five graders from this school had the lowest initial digital literacy scores, this finding suggest that a certain starting level of digital literacy is necessary for the practice to be effects. Contrary, the use of the practice by insufficiently skilled students could have negative effects, probably by affecting their perceived competence during the learning activity. School B, which had technical obstacles that caused interruptions and prevented an autonomous use of the game by students, had mixed results. The practice had impact on motivation in grades 5 and 8 (p<.05), while no difference was found in grades 6 and 7 (p>.05). The practice didn’t show effects on learning outcomes measured by the digital literacy test in school A (p between .159 and .922). However, in school B, where students experienced technical difficulties while using the game, the control group had better achievement on the posttest measure than their experimental group counterparts (p<.05). This finding stressed the importance of ensuring the proper technical conditions prior to implementing the practice. Students’ and teachers’ insights revealed enablers and barriers to the practice implementation and supported fine nuancing of the quantitative findings, thus enlightening the mechanisms through which the practice impacted learning, which is applicable to other ICT based teaching resources as well.", journal = "European Conference on Educational Research 2024, Nicosia, Cyprus", title = "Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons", url = "https://hdl.handle.net/21.15107/rcub_reff_6639" }
Mićić, K., Veljković, K., Stančić, M.,& van Uden, J.. (2024). Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons. in European Conference on Educational Research 2024, Nicosia, Cyprus. https://hdl.handle.net/21.15107/rcub_reff_6639
Mićić K, Veljković K, Stančić M, van Uden J. Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons. in European Conference on Educational Research 2024, Nicosia, Cyprus. 2024;. https://hdl.handle.net/21.15107/rcub_reff_6639 .
Mićić, Katarina, Veljković, Katarina, Stančić, Milan, van Uden, Jolien, "Digital Literacy through Games: A Participatory Assessment Study of the Impact of a Minecraft-Based Learning Resource for Computer Science lessons" in European Conference on Educational Research 2024, Nicosia, Cyprus (2024), https://hdl.handle.net/21.15107/rcub_reff_6639 .