Data Collection

Foertsch (1998) provides the following example of data collection from an evaluative study of one school district's reading program. The example shows the preplanning involved in data collection and indicates how to ensure that this phase of the process is directly related to research questions.

Phase 1: Data Collection Instruments

"In conjunction with administrators from District 31, our research team developed a set of questions and issues to discuss at group and individual interviews, wrote protocols to ensure consistency in data collection activities during classroom observations, and developed a teacher/staff questionnaire related to reading practices and instructional approaches.

Six primary research questions guided us in creating the data collection instruments and in analyzing the data. The questions are as follows:

1. What are the critical features of District 31's reading programs?
2. What contextual factors impact the implementation of these programs?
3. What strategies within the District are available for supporting the implementation of the programs?
4. How consistently are programs being implemented within and across grade levels?
5. What is the impact of various program components on student achievement in reading?
6. How well is reading achievement currently being measured?

Data Collection Instruments. The following data collection instruments were developed for this study:

• Group Interviews with Teachers. To identify the major issues surrounding the planning and implementation of reading instruction in the district, group interviews were conducted with reading/language arts teachers and staff at both the elementary and middle schools in District 31. An interview protocol was used to facilitate discussion. Responses to the uniform standard questions contributed to the final analysis of the study findings.

• Individual Interviews with Teachers. To investigate all of the major research questions presented earlier, individual 20-minute interviews were conducted with teachers and staff at the elementary and middle schools. An interview protocol was used to facilitate information gathering. Responses to the questions contributed to the final analysis of the study findings.

• Teacher/Staff Questionnaire. A teacher/staff questionnaire designed to gather information relevant to the first two research questions was administered to all elementary and middle school teachers in the district. The first part of the questionnaire focuses on general information related to school climate. The second part of the questionnaire focuses on program-specific information by asking reading/language arts/humanities teachers to indicate whether or not certain program characteristics are present in their instruction, and the extent to which they are present (amount of time spent on specific tasks). We also gathered information about the extent to which reading is part of content-area instruction in grades 3, 6, and 8.

  Program-specific characteristics were viewed from the lens of best practices in reading that have been identified in the research literature include the following:

  • The teaching of reading and writing as processes.
  • The integration of reading and writing across the curriculum.
  • Use of discussion to promote literate behaviors.
  • Use of collaborative activities.
  • A balanced approach to instruction.
  • Breadth and depth of reading experiences.
  • Independent reading.
  • Focus on reading for meaning.
  • Evaluation that focuses on processes and informs instruction.
  • Vision/philosophy of reading and reading instruction.
  • Alignment of assessment with instruction.

  To the extent possible, other nonprogram-specific characteristics that have been demonstrated to impact student achievement were also examined, including:

  • Resources for accommodating ESL and special-needs students.
  • Program and staff support.
  • Professional development.
  • Program effectiveness from the teachers' point of view.

Classroom Observation Protocol. Reading instruction was observed in classrooms across all grade levels (K-8, ESL/bilingual, special education, Title I) in order to better understand how instruction is actually delivered. Observation notes were analyzed according to major categories contained in the protocol to reveal themes. These themes were categorized initially as strengths, issues, and opportunities for growth. Additional data collection and review occurred during the observations. Researchers summarized notes after each observation.

Document Review. Document review informed the research process to varying degrees. Among the documents analyzed were student work samples, report card forms, letters to parents, lesson plans, lists of reading books, and other relevant documents pertinent to current reading instruction." (pp. 16-18)

References