Evaluating Technology-Integrated Education
One of my areas of interest as a UX researcher are discovering the needs of the education-professional user. I decided to investigate what research has been conducted around how educators feel about the use of technology in the classrooms, such as 1:1 laptop-to-student programs or educational software. The following is a summary of my literature review that explores how educators and administrators evaluate the effectiveness of technology-integrated education.
What are the requirements for assessing technology-integrated education? What are the existing evaluation practices and user-perspective assessments?
- Literature Review
The objective of this paper is to explore the methods used for evaluating technology-integrated environments to determine the best practices for assessments. In order to create the criteria for successful evaluation, I researched current assessment tools that are designed to be used across diverse contexts. I also examined evaluation methods that measure the needs of the primary users, the students and teachers.
Based on these sources, I concluded that successful evaluation requires standardized criteria and goals in order for researchers and practitioners to align their objectives for using technology-integrated education. Additionally, I found that successful evaluations must include users’ perspectives; this information gives context for how technology affords educational opportunities, and where it could improve the experience.
This research is helpful for understanding the objectives when implementing technology-integrated educational programs. The study is also a resource for administrators and policy makers to understand role that technology plays within academic performances.
The demand for technology-integrated education exceeds the ability to implement effective programs. Educators take a trial-and-error approach in their classrooms to establish the best methods; however, in a world that rapidly increases reliance on technology, students cannot afford being on the error side of the process. Central to the issue of integrating technology into classrooms and curriculum is that teachers, administrators, parents and students lack the tools needed to determine what works and what needs to be improved. Without these tools, the needs of these users are not addressed.
My aim in this paper is understand the requirements for assessing technology-integrated education by exploring existing evaluation practices and user-perspective assessments
Benefits of Technology in Education
First of all, in order to understand what is needed to create evaluation methods, I examined the benefits of technology in education.
Information and Communications Technology
- Information and Communications Technology (ICT) allows for up-to-date learning and teaching that prepare students for real life and their futures.
- Ubiquitous learning (or u-learning) is used in education “to construct a ubiquitous learning environment, which enables anyone to learn at anyplace at anytime”.
Next I looked at existing comprehensive evaluation models.
- The indicator system evaluates ICT based on two domains: the environment and application of ICT in education. The researchers concluded that by adjusting evaluations based on these two domains, an indicator systems could be applied across many organizations and cultures.
- The second model I looked at used five-learning scenarios to evaluate technology-integrated classrooms. These five scenarios served as objectives for the use of technology in classrooms. For example, one scenario looks at if the curriculum objective matches the participant’s perceptions of the activities.
User Perspectives and Needs
I then looked at how users were assessing the use of technology in the classrooms.
Factors of teacher use
- First, the research of Chiero determined the factors of whether or not a teacher used computers in teaching. The three major factors were if there was technology support available; how long it took to learn new software; and “the availability of a district computer coordinator”.
Features important to the User
- Shee and Wang looked at what was important in technology integrated education from the students perspectives. They found that the learners deemed the most important dimension to be the learner interface, because this is where a “high level of interaction takes place”.
- Overall, having a shared understanding of the criteria and goals of evaluation allows researchers and practitioners to align their objectives for using technology-integrated educations.
- The evaluation method must include assessing the needs of the users. Asking the users for their input during an evaluation adds assurance that their concerns were taken into consideration and changes made will benefit their experience. This information also gives additional context to these quantitative assessments, which is helpful in finding the source of issues that students and teachers may be experiencing.
Conclusions and Future Work
- Based on the research cited in this paper, a successful evaluation method of technology-integrated classrooms is one that can be applied to various contexts while also directly addressing the needs of the users that directly interact with them. These two areas are essential for developing programs to meet the demand of technology-integrated education.
- More research should be done into how and if technology can serve as a suitable replacement to evaluating student work and perhaps teacher performance as well
- Further studies should research on how to incorporate statewide policies and standards, thus appealing to secondary users, administrators and teachers.
CLICK HERE to download the full research review paper (in APA Format!)
- Barrell, B. (Ed.). 2001. Technology, teaching and learning: Issues in the integration of technology. Calgary, Alberta: Detselig Enterprises.
- Chiero, R. T. (1997). Teachers' perspectives on factors that affect computer use. Journal of Research on Computing In Education, 30(2), 133.
- Huang, Y., Chiu, P., Liu, T., & Chen, T. (2011). The design and implementation of a meaningful learning-based evaluation method for ubiquitous learning. Computers & Education, 2291-2302.
- Kompf, M. (2005). Information and Communications Technology (ICT) and the Seduction of Knowledge, Teaching, and Learning: What Lies Ahead for Education. Curriculum Inquiry, 35(2), 213-234.
- Ma, W., & Li, S. (2013). An Exploration of Student Satisfaction in Online Accounting Courses. Hybrid Learning and Continuing Education Lecture Notes in Computer Science, 8038, 155-166.
- Rouse, M. (n.d.). What is ICT (information and communications technology - or technologies)? -Definition from WhatIs.com. Retrieved May 21, 2015, from http://searchcio.techtarget.com/definition/ICT-information-and- communications-technology-or-technologies
- Sakamura, K., & Koshizuka, N. (2005). Ubiquitous Computing Technologies for Ubiquitous Learning. IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE'05), 11-20.
- Shee, D., & Wang, Y. (2008). Multi-criteria Evaluation Of The Web-based E-learning System: A Methodology Based On Learner Satisfaction And Its Applications. Computers & Education, 50(3), 894-905.
- Tang, C., & Yu, Y. (2013). An Exploratory Study on Instructors’ Agreement on the Correctness of Computer Program Outputs. Hybrid Learning and Continuing Education Lecture Notes in Computer Science, (8038), 69-80.
- Yahya, S., Arniza Ahmad, E., & Abd Jalil, K. (2010). The definition and characteristics of ubiquitous learning: A discussion. International Journal of Education and Development Using Information and Communication Technology (IJEDICT), 6(1), 117-127.
- Zhao, J., & Jiang, Y. (2013). Developing an Indicator System of ICT in Education: From Conceptual Model to Items Extraction. Hybrid Learning and Continuing Education, 8038, 110-121. doi:10.1007/978-3-642-39750-9_11