This study examines the impact of integrating mobile technology with audience response systems (ARS) on students’ performance and experience. By comparing students in classes using traditional single-purpose ARS (Clicker) with students in classes using mobile technology-based ARS (Mobile ARS), we find that the latter earned, on average, 3.6 percent more on their final examinations and reported a more positive experience in three different financial accounting courses. Our findings suggest that the benefits of ARS technology documented by prior studies not only survived but were also strengthened by combining ARS technology and mobile technology. Leveraging the two technologies within the classroom environment appears to provide an advantageous platform that improves students’ grade performance and classroom experience.
The positive impact of audience response systems (ARS) on students’ performance and experience has been well established in the ARS literature. Most, if not all, of such empirical evidence is based on traditional single-purpose ARS (Clicker), which is used exclusively for participation in Clicker-related classroom activities1 . With the rapid advances in mobile technology and the increasingly ubiquitous presence of mobile devices, Clicker has also gone mobile.2 However, whether mobile technology-based ARS (Mobile ARS) is as effective as Clicker as a classroom teaching tool remains unknown. On the one hand, combining mobile technology and ARS could enhance the positive impact of ARS because mobile technology renders ARS more attractive and more interesting (e.g., touch screens, background music, and rankings of participants). On the other hand, mobile devices are known to be a huge distraction in classrooms.3 When permitted to use mobile devices for ARS activities, students may use mobile devices more often for class-unrelated activities, leading to a decrease in the positive impact of ARS. Could Mobile ARS be an advantageous learning platform that retains the positive impact of Clicker despite the distractive effect of mobile devices or would the distractive effect of mobile devices diminish the positive impact of Clicker once the two technologies are integrated? This study aims to answer these questions by examining the impact of integrating mobile technology and ARS on students’ experience and performance in financial accounting courses.
We conducted a controlled experiment in three different financial accounting courses. Each course had two sections (classes) offered in the same semester that were taught by the same instructor with the same structure and contents, except for that Mobile ARS was used for regular quizzes in one section, and Clicker was used in the other section. Our goal was to compare the impact of Mobile ARS with that of Clicker on student performance, which was proxied by their final exam grades, and student experience, which was based on a course-end survey. We found that the students in the Mobile ARS sections earned, on average, 3.6 percent more on their final examinations and reported a better experience with ARS than the students in the Clicker sections. These results suggest that the combination of mobile technology and ARS technology creates an advantageous classroom learning platform and that the positive effect of ARS is strengthened by mobile technology.
This study makes three important contributions. First, this study fills the gap in ARS research by providing evidence regarding the effect of the current iteration of ARS technology, i.e., Mobile ARS. Most studies investigating ARS have been conducted using Clicker as the delivery instrument. However, with advances in mobile technology, Clicker has become a thing of the past. To the best of our knowledge, few studies have focused on the impact of Mobile ARS. Thus, whether the positive impact of Clicker survives ARS’ transition to mobile technology is undocumented and unknown.
Second, this study contributes to research concerning the impact of mobile technology on education by providing an example of the beneficial combination of mobile technology with other classroom technologies. Despite the immense potential of mobile technology (Cerratto, Pargman, Nouri and Milrad, 2018; Li and Song, 2018; Ryberg, Davidsen and Hodgson, 2018; Schachter, 2009; Sun, Lin, Wu, Zhou, and Luo, 2018), mobile devices have become known more as a source of substantial problem in education due to student’s excessive preoccupation with them. A survey conducted by Tindell and Bohlander (2012) investigating mobile device use in college classrooms revealed that 95 percent of students brought their phones to class daily and that 92 percent of students used their phones to send text messages during class time. Furthermore, the students became distracted by other students’ texting during class. Duncan, Hoekstra, and Wilcox (2012) established a negative association between in-class use of mobile devices and student performance. Specifically, students who reported regular cell phone use in class showed an average negative grade difference of 0.36 on a four-point scale of final grades in eight introductory science courses at a major U.S. university. Despite the intense debate regarding the effects of mobile devices on student classroom performance and experience, both proponents and opponents of this technology agree that mobile devices are here to stay and that keeping them out of classrooms is practically impossible. The findings that ARS mobilization strengthens the existing positive impact of Clicker reinforces the notion that if one cannot fight this technology, it is better to use it wisely.
Finally, this study provides a new perspective regarding the evaluation of classroom technologies. Most existing studies investigating classroom technologies focus on the effects of a single technology in isolation. The recent trend in consumer electronics development suggests that multifunctioning devices are rapidly replacing less integrated devices. Research concerning classroom technologies should consider the potential interaction of different technologies. To the best of our knowledge, this study is the first to focus on the impact of classroom technology integration.
The fact that accounting courses were chosen for our experiment also gives this study added relevance. In recent decades, there has been a decline in the quantity and quality of students who chose to study accounting, especially in the U.S., Canada and the UK (Salemi & Eubanks, 1996). Financial accounting courses are cumulative in nature and particularly benefit from the active learning approach. Effective teaching tools, such as ARS, which have a potential positive impact on students’ experience and performance through better engagement, can help students succeed in their courses and stay in accounting programs. This study provides evidence that Mobile ARS is even more effective than Clicker in this regard.
The remainder of this paper is organized as follows. In the following section, we provide a review of the relevant literature and develop the two study hypotheses. Then, we introduce our research design and provide a description of our data. Next, we present the results of our hypothesis tests, and finally, we summarize and provide the conclusions of the study.
2. Literature review and hypothesis development
Previous studies have shown that ARS can improve students’ performance and learning experiences in a wide range of courses (e.g., Anderson et al., 2015; Bode, Drane, Kolikant, & Schuller, 2009; Edmonds & Edmonds, 2008; Oswald & Rhoten, 2014; Premuroso, Tong, & Beed, 2011; Stowell & Nelson, 2007; Sprague & Dahl, 2010). For example, Edmonds and Edmonds (2008) provided evidence that on average, students using ARS earned 3.15 percent higher course grades than students who did not use ARS. Chui, Martin, and Pike (2013) found that students in ARS sections received higher quiz scores than those in sessions without ARS, although there was no significant difference in the two groups’ final grades. These authors suggested that immediate feedback caused the positive effects of ARS on student performance.