          |
A Meta-Analysis of the Effectiveness of Teaching and Learning With Technology on Student OutcomesResultsThe results comprise two sections. The first section summarizes the quantity, type, and quality of studies included in the review. The second section summarizes the overall findings from the studies. Description of Studies in Review Initially, a total of nearly 200 potentially applicable articles were retrieved. Upon further application of the criteria for the synthesis, however, only 42 articles were included in the final synthesis. Many of the articles were eliminated because they did not provide the relevant statistics for calculating effect sizes. Other studies were eliminated because students in the control groups had access to or used technology. The final sample of studies included 42 journal articles. A total of 282 effect sizes were calculated from the 42 studies. The studies contained a combined sample of about 7,000 students. The mean number of students in the sample of studies was 184, and the range was from 32 to 2,802. About half of the studies had sample sizes of less than 50, and only 25 percent of the studies had sample sizes greater than 100. About 20 percent of the studies were published in 2001, 15 percent in 1999, 14 percent in 2000, and 12 percent for 2002 and 2003. The average number of comparisons in each study was approximately 7, but the range was from 1 to 27. About 40 percent of the studies focused on elementary school (Grades K–5), 40 percent on middle-level school (Grades 6–8), and 20 percent on secondary school (Grades 9–12). In terms of research design, about 67 percent of the studies included in the synthesis were quasi-experimental, using either a nonrandomized static-group posttest comparison design (19 percent), a nonrandomized one-group pretest-posttest design (21 percent), or a nonrandomized pretest-posttest control group design (25 percent). Only 25 percent of the studies used an experimental (randomized) pretest-posttest control group design or a randomized posttest-only control group design. In terms of type of technology, 30 percent of the studies used personal computers, 26 percent used networked laboratories, 5 percent used multimedia, and the other 39 percent used a variety of other technology resources. In terms of instructional software, 31 percent of the studies used an exploratory environment such as simulations, hypermedia, and hypertext. About 10 percent used drill-and-practice software, 7 percent used tools for other tasks such as word processing or e-mail, and 32 percent used mixed forms of technology. About 20 percent of the studies did not specify the software they used. Evidence for the use of Five Standards for Effective Pedagogy was not very prevalent in the studies reviewed. In 71 percent of the studies, for example, there was no evidence that instructional conversations (extended dialogue between teachers and students) occurred in the classroom. In more than half of the studies, the use of language and literacy activities, contextualization/making meaning, and challenging activities was not described. The one standard that was somewhat prevalent was joint productive activities. There was some evidence that it was prevalent in 34 percent of the studies, and there was extensive evidence that it was prevalent in 21 percent of the studies. The cognitive outcomes used in the 42 studies varied widely. The most common cognitive outcomes were a researcher-based test (38 percent), followed by authentic assessments (14 percent), and then standardized tests (10 percent). About 57 percent of the affective outcomes were student attitudes towards computers, and 18 percent were students' motivation or self-concept. About 83 percent of the behavioral outcomes examined in the studies in this synthesis focused on the number of tasks attempted, followed by student time-on-task (18 percent), and student perseverance (5 percent). Overall Results Table 2a and Table 2b list the mean study-weighted means and the unweighted means for each of the three outcomes and the overall mean. The standard deviations, confidence intervals, and number of comparisons also are included in Table 2a and Table 2b. The mean of the study-weighted effect sizes averaging across all outcomes was .410 (p < .001), with a 95-percent confidence interval of .175 to .644. This result indicates that teaching and learning with technology has a small, positive, significant (p < .001) effect on student outcomes when compared to traditional instruction.
In addition to examining the overall mean study-weighted effect size, we also examined the effect sizes for each of the three types of outcomes. The mean study-weighted effect size for the 29 study-weighted comparisons containing cognitive outcomes was .448, (p < .01), with a 95-percent confidence interval of .171 to .724. This result indicates that teaching and learning with technology has a small, positive effect on students' cognitive outcomes when compared to traditional instruction. The mean study-weighted effect size for the 10 study-weighted comparisons that focused on student affective outcomes was .464. (p > .05), with a 95-percent confidence interval of -.166 to 1.094. This result indicates that teaching and learning with technology has a small, positive, non-significant (p > .05) effect on students' affective outcomes when compared to traditional instruction. Finally, the mean study-weighted effect size for the three study-weighted comparisons that contained behavioral outcomes was -.091, (p > .05), with a 95-percent confidence interval of -.142 to 1.243, indicating that technology had a slight, negative, nonsignificant effect on students' behavioral outcomes. The unweighted effect sizes are similar to the study-weighted results. Of the 282 effect sizes that were examined in the 42 studies, about 71 percent were positive. The overall, unweighted effect size was .414, (p < .001), with a 95-percent confidence interval of .332 to .497. The unweighted effect size was .544 for cognitive outcomes, .290 for affective outcomes, and .087 for behavioral outcomes. The standard deviations for both the study-weighted and unweighted effect sizes are quite large, indicating a great deal of variation among the studies. The confidence intervals reported in Table 2a and Table 2b describe the precision of the estimate of the mean effect size by indicating the range within which the population mean is likely to be, given the observed data (Lipsey & Wilson, 2001). The relationship of each of the 57 conditioning (i.e., independent) variables to the mean study-weighted effect size was tested for significance using ANOVA. The results indicate that none of the variables had a statistically significant (p < .01) impact on the study-weighted effect size. In other words, the overall findings suggest that the results do not differ significantly across categories of technology, instructional characteristics, methodological rigor, characteristics of the study, and subject characteristics.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
 Contact Us | Privacy Policy |