Skip over navigation
Learning Point Associates Logo
North Central Regional Educational Laboratory
NCREL HomeNCREL Sitemap
Photo of children and teachers using technology
Technology Home
Assessment and Evaluation
Research
E-Learning
Planning
Policy
Professional Development
Research
Teacher Preparation
Teaching and Learning
Technology in Education
SRI International

An Educator's Guide to
Evaluating Claims about Educational Software

Red Flags Introductions Checklist Resources Glossary Buyer's Worksheet
Checklist

Does this research finding apply to my classroom, school, or district?
How do I determine if the research is designed to support these claims?

 

[comparison group issues]
[pre-post design]
[sample size issues]
[interpreting results]
How is student achievement measured?
Summary

How do I determine if the research is designed to support these claims?

To conclude that software or technology was the cause of certain results, researchers need to design a study according to certain standards. Two important design standards are to use a comparison group and to recruit a large enough sample.

In a comparison group design, researchers attempt to compare the performances of students with the technology to performances of students without the technology or a different type of technology to find out the impacts of the treatment. However, to do so reliably, researchers need to recruit a large enough sample to make inferences based upon statistical methods (see inferential statistics). Generally speaking, there should be at least 30 members in each group that you are studying, but this number is a bare minimum, and larger numbers improve accuracy.

Consider these questions when reviewing a study:

Where can you find this information?
You can find information about study design and sample size in a study's Methods section. Sometimes, the size of the sample can be found in tables in the Results section or in the Appendix.

Example Study

Methods

Academic achievement was compared between 1,300 students enrolled in 20 schools participating in Computer Reading Fun and 500 students in 10 nonparticipating schools. The students in the participating and nonparticipating schools were of similar ethnic composition and socioeconomic status.

Results

Students in schools with more experience using Computer Reading Fun reported average gains in math achievement of 6.65 NCE points and a 10-point increase in students' percentile rank, which was significantly higher than the changes in the nonparticipating schools.



Copyright © SRI International. All rights reserved.
Please send comments and suggestions.

This site was created by the Center for Technology in Learning at SRI International under a task order from the Planning and Evaluation Service, U.S. Department of Education (DHHS Contract # 282-00-008-Task 3).

Center for Technology in Learning

Last updated on: 11/04/02

 


Contact Us | Privacy Policy
Copyright © Learning Point Associates.
All rights reserved.
Disclaimer and copyright information.