Guidelines for Evaluating Professional
Development Guskey (1998) suggests the following guidelines for evaluating professional development:
"1. Clarify the intended goals. The first step in any evaluation is to make sure your professional development goals are clear, especially in terms of the results you hope to attain with students and the classroom or school practices you believe will lead to those results. Change experts refer to this as 'beginning with the end in mind.' It is also the premise of a 'results-driven' approach to professional development (Sparks, 1995, 1996b).
2. Assess the value of the goals. Take steps to ensure the goals are sufficiently challenging, worthwhile, and considered important by all those involved in the professional development process. Broad-based involvement at this stage contributes greatly to a sense of shared purpose and mutual understanding. Clarifying the relationship between established goals and the school's mission is a good place to begin.
3. Analyze the context. Identify the critical elements of the context where change is to be implemented and assess how these might influence implementation. Such an analysis might include examining pertinent baseline information on students' and teachers' needs, their unique characteristics and background experiences, available resources, parent involvement and support, and organizational climate.
4. Estimate the program's potential to meet the goals. Explore the research base of the program or activity, and the validity of the evidence supporting its implementation in contexts similar to yours. When exploring the literature on a particular program, be sure to distinguish facts from persuasively argued opinions. A thorough analysis of the costs of implementation--and what other services or activities must be sacrificed to meet those costs--should be included as well.
5. Determine how the goals can be assessed. Decide up-front what evidence you would trust. Ensure that evidence is appropriate, relevant to the various stakeholders, and meets at least minimal requirements for reliability and validity. Keep in mind, too, that multiple indicators will probably be necessary, in order to tap both intended and possible unintended consequences.
6. Outline strategies for gathering evidence. Determine how evidence will be gathered, who will gather it, and when it should be collected. Be mindful of the critical importance of intermediate or benchmark indicators that might be used to identify problems (formative) or forecast final results (summative). Select procedures that are thorough and systematic, but considerate of participants' time and energy. Thoughtful evaluations typically use a combination of quantitative and qualitative methods, based on the nature of the evidence sought. To document improvements, you must also plan meaningful contrasts using appropriate comparison groups, pre- and post-measures, or longitudinal time-series measures.
7. Gather and analyze evidence on participants' reactions. At the completion of both structured and informal professional development activities, collect information on how participants regard the experience. A combination of items or methods is usually required to assess perceptions of various aspects of the experience. In addition, keeping the information anonymous generally guarantees more honest responses.
8. Gather and analyze evidence on participants' learning. Develop specific indicators of successful learning, select or construct instruments or situations in which that learning can be demonstrated, and collect the information through appropriate methods. The methods used will depend, of course, on the nature of the learning sought. In most cases, a combination of methods or procedures will be required.
9. Gather and analyze evidence on organizational support and change. Determine the organizational characteristics and attributes necessary for success, and what evidence best illustrates those characteristics. Then collect and analyze that information to document and improve organizational support.
10. Gather and analyze evidence on participants' use of new knowledge and skills. Develop specific indicators of both the degree and quality of implementation. Then determine the best methods to collect this information, when it should be collected, and how it can be used to offer participants constructive feedback to guide (formative) or judge (summative) their implementation efforts. If there is concern with the magnitude of change (Is this really different from what participants have been doing all along?), pre- and post-measures may need to be planned. The methods used to gather this evidence will depend, of course, on the specific characteristics of the change being implemented.
11. Gather and analyze evidence on student learning outcomes. Considering the procedures outlined in Step 6, collect the student information that most directly relates to the program or activity's goals. Be sure to include multiple indicators to tap the broad range of intended and possible unintended outcomes in the cognitive, affective, and psychomotor areas. Anecdotes and testimonials should be included to add richness and provide special insights. Analyses should be based on standards of desired levels of performance over all measures and should include contrasts with appropriate comparison groups, pre- and post-measures, or longitudinal time-series measures.
12. Prepare and present evaluation reports. Develop reports that are clear, meaningful, and comprehensible to those who will use the evaluation results. In other words, present the results in a form that can be understood by decision makers, stakeholders, program developers, and participants. Evaluation reports should be brief but thorough, and should offer practical recommendations for revision, modification, or further implementation. In some cases, reports will include information comparing costs to benefits, or the 'return on investment.' " (pp. 41-43)
