NCREL Policy Issues
Issue 13, December 2002
Building Student Achievement: In-School and Out-of-School Factors
By Reginald Clark, Ph.D.
There is much that is not known about the actual lives of academically successful youths, especially those successful youths from impoverished backgrounds. Researchers have conducted few studies of achievement patterns among the same cohort of urban students that adequately take into account the role of family, school, and neighborhood process factors. That is, studies rarely analyze students' school achievement patterns in relation to the students' daily and weekly activities and routines and their overall lifestyles. Data presented in this paper show that variations in students' achievement test scores are closely associated with actions that are taken by students, teachers, parents, and others in pursuit of achievement. Data from four samples drawn from exploratory research studies are discussed in this paper. There are three elementary school samples (Nashville, Tennessee; Bakersfield, California; and Los Angeles, California) and one high school sample (Long Beach, California). Exhibit 1 below lists pertinent information about each sample.
Exhibit 1
|
Factors That Contribute to Student Achievement
In my most comprehensive study to date, researchers analyzed data gathered from 459 elementary school students, their parents, and teachers in five schools in Nashville, Tennessee. A student was included in the survey if one of his or her parents responded to a parent questionnaire that was sent home from the school, and if the student completed a survey about his or her weekly time use. This sample consisted of 247 female and 212 male elementary students. Teachers of all 459 students completed a teacher survey (n=19). For analysis purposes, each teacher's responses were linked to the time-use data and the parent survey data for each one of their specific students.
There were significant race/ethnicity and income differences among the student population. The majority of the students were White (57 percent). Blacks composed 33 percent of the sample. The rest of the sample consisted of Asians, Latinos, and "others" (10 percent). For this study, student participation in a federally sponsored free or reduced-price lunch program was used as an indicator of social class status. Eighteen percent of the White students were receiving free or reduced-price lunches. More than three times as many of the Black students (58 percent) were receiving free or reduced-price lunches at the school.
There were achievement test performance gaps relating to socioeconomic status (SES) and race in the sample. With regard to SES, twice as many "lower-achiever" students were receiving support from the federal school lunch program. Approximately 48 percent of the lower achievers and 24 percent of the "high-achiever" students were receiving free or reduced-price lunches at the school. (In the Nashville study, high achievers were students whose normal curve equivalency [NCE] scores on the reading total portion of the Tennessee Comprehensive Assessment Program were at or above the 50th percentile. Lower-achieving students had scores below the 50th percentile.) Race disparities in achievement test scores also were apparent. The reading total median NCE score was 65.3 for the White students in the sample and 44.1 for the Black students. A total of 277 students (60 Black, 192 White, and 25 "other") had test scores ranking them as high achievers, while 182 students (93 Black, 71 White, and 18 "other") had scores in the lower-achievers category.
As a result of these racial variations in achievement, the Nashville data provide a rich opportunity to assess what factors may contribute to creating the achievement gap between higher- and lower-achieving students from different social class and racial groups. The findings potentially have great utility in identifying practices (processes) that can impact the narrowing of the achievement gap. From this information, policies can be proposed to address the educational needs of lower-achieving students, the majority of whom have lower-income and ethnic-minority status (Black, Asian, Latino).
In analyzing this data set, correlation and multiple-regression analysis methods were used to explain variations in students' scores on standardized tests of reading. All scores were converted into standardized Z scores to conduct the analysis. The correlation analysis revealed profoundly higher relationships for instructional-process factors (such as teacher estimate of student time on classroom learning and teacher perception of student capabilities, teacher-parent communication patterns, parental standards for student academic pursuits, and students' patterns of out-of-school time use) than for noninstructional factors (such as family income, whether or not income is from government aid, and participation in a free lunch program). In fact, the instructional-process factors explained far more of the total variance in students' academic scores than family ethnicity, economic circumstances, and perceived safety level in the community of residence combined.
The results of the analysis revealed that about 51 percent of the variation in student test scores was accounted for by school-process factors and family-process factors. Exhibit 2 shows that when instructional-process factors are taken into account, student ethnicity and parent socioeconomic status are nearly eliminated as impacts on student achievement. Indeed, beta scores on the family background factors (ethnicity and socioeconomic status) are negatively correlated with students' scores on the Tennessee Comprehensive Assessment Program test of reading, after taking into consideration the pertinent school-process factors and out-of-school family and time-use factors. Similarly, beta scores on the community-safety variable (as perceived by parents) independently contributed less than 10 percent to the variation in students' test scores in reading.
Exhibit 2
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
These findings, in combination with findings in the sections below, suggest that the factors that matter most for student achievement on standardized tests are as follows: teacher instructional actions and expectations for students; students' total weekly out-of-school time in high-yield activities; activity quality; parental standards, beliefs, and expectations; and teacher-parent communication actions.
Teacher Instructional Actions and Expectations for Students
Data from the Nashville and Bakersfield, California, studies were used to assess the role of teacher classroom actions on student achievement. In the Nashville study, 19 elementary school teachers of 459 first- through sixth-grade students responded to the following questions:
- On an average day, how many hours or minutes do you think your students are actively engaged in learning in your classes?
- What percentage of the poor-reader students in your classes have the biological capability to one day attend and complete college?
- What percentage of the poor-reader students in your class would you say want to go to college?
A multivariate analysis showed that more than one-fifth of the variance in students' reading achievement scores (22 percent) was accounted for by teacher responses to these three questions. Higher-achieving students were more likely to have a teacher who provided more exposure to classroom lessons ("time-on-task"), who believed the majority of lower-achieving students in her class had the biological capability to one day attend and complete college, and who believed the majority of her lower-achieving students wanted to go to college one day.
The Bakersfield, California, study offers more evidence of the strong impact of teacher instructional actions in the classroom. This study tested the hypothesis that high-achieving students spend more time than low-achieving students learning academic lessons in the classroom. Videotapes were made of students in 13 first- through third-grade classrooms in five Bakersfield elementary schools. The racial composition of these classrooms and schools, which was representative of the Bakersfield first-through third-grade student population, was about 40 percent Latino (Mexican American), 40 percent White, 15 percent African American, and 5 percent Asian American (Southeast Asian).
A video camera was set up in a corner in each of the 13 classrooms. The camera taped the activities of the students and teachers throughout one 6-hour day of classes. The tapes were later analyzed to determine learning time and class time. Researchers identified one African-American student in each classroom and timed his or her activities using a stopwatch. When the student appeared to be engaged in learning activities (such as reading, working alone on a lesson, listening to a lecture, solving a problem with classmates, or asking questions), the stopwatch was turned on. When the student was off-task or involved in behaviors that were not learning activities, the stopwatch was turned off until the student started another learning activity.
NCE scores for reading on the Comprehensive Test of Basic Skills (a standardized basic skills test) later were gathered for each of the 13 observed students. Nine of the target students had an NCE score below the 20th percentile. These students were labeled "poor readers." Four of the target students had an NCE score between the 35th and 45th percentile; they were assigned the designation of "average readers." Exhibit 3 shows that although both groups of students had almost the same amount of available class time, average readers were involved in classroom learning 1 hour and 47 minutes more each day than poor readers. Average readers spent 3 hours and 41 minutes engaged in daily learning while poor readers spent only 1 hour and 54 minutes in these same activities.
|
Students' Total Weekly Out-of-School Learning Time and Activities
Two key hypotheses are pertinent here. The first hypothesis is that high-achieving students spend more time engaged in academic lessons in the classroom than low-achieving students and they spend more time (hours per week) engaged in structured out-of-school literacy-enhancing activities. Second and conversely, low-achieving students spend less total time engaged in structured learning activities (which includes combined in-school and out-of-school time).
The data show that at the elementary and high school educational levels, high achievers spent more time in out-of-school high-yield learning activities than low achievers. High-yield out-of-school learning activities include such diverse activities as leisure reading, writing, studying, getting tutored, participating in community and school youth clubs and programs, working on the computer, watching educational television, volunteering, doing hobbies, and playing organized youth sports. The time spent by students in these activities is an indicator of the extent of their learning activities outside of school.
In particular, better readers spent more out-of-school time involved in powerful, high-impact (high-yield), language-enriched activities that promote successful acquisition and expansion of developmentally appropriate reading skills. These activities included the following:
- Weekly time dialoguing with adults, youth club enrichment activities, hobby and volunteer activities, organized sports, and educational television.
- Regular study and homework routines, often with adult or peer monitoring and support.
- Reading and writing practices in the home, sometimes including composing text on the computer.
In the Nashville elementary school sample, high-achieving students spent an average of 7 hours and 56 minutes per week engaged in out-of-school learning activities, while low-achieving students spent only 7 hours per week engaged in out-of-school learning activities. This difference in time was not statistically significant.
Similar group differences were found in the sample of 11th-grade high school students in Long Beach, California. Based on scores on a district-approved writing test, 20 high school students were classified as high achievers and 30 were classified as lower achievers. The high achievers were using more of their out-of-school time in learning activities than the low achievers. High-achieving high school juniors spent 15 hours and 14 minutes per week doing learning activities outside of school. Their low-achieving counterparts spent much less time—8 hours and 49 minutes per week—doing these activities. (The time difference between the two groups equals 6 hours and 25 minutes per week.)
The second hypothesis is that low-achieving students spend less total time engaged in structured learning activities than do high-achieving students. Unstructured leisure activities include but are not limited to hanging out and playing, talking on the telephone, playing video games, using the computer for fun, playing board games, watching television or movies, listening to music, attending sports events, resting, or relaxing. Data from the Nashville elementary school sample support this hypothesis. In the Nashville sample, student time spent in leisure activities was negatively correlated with achievement. That is, high-achieving students spent less time engaged in unstructured leisure activities than did low-achieving students. High-achieving students spent an average of 27 hours and 13 minutes per week in unstructured activities. Low-achieving students spent 28 hours and 45 minutes per week in these activities.
The Significance of Total Weekly In-School and Out-of-School Learning Time
These data consistently show that high achievers at the elementary and high school levels spent more time in weekly learning activities than their low-achieving counterparts. Such learning activities include activities in school and in out-of-school enrichment situations. Exhibit 4 shows that high-achieving first through sixth graders spent a total of 44 hours and 40 minutes per week doing weekly learning activities (in-school and out-of-school), while low-achieving first through sixth graders spent 42 hours and 34 minutes per week doing these same activities. (The difference between the two groups equals 2 hours and 6 minutes per week.) Total weekly learning time was positively correlated with achievement for the elementary students.
Following the same pattern, the high-achieving high school juniors in Long Beach, California, spent 47 hours and 37 minutes per week doing learning activities (in-school and out-of-school) while low-achieving high school juniors spent 40 hours and 15 minutes per week in these activities. (The difference between the two groups equals 7 hours and 22 minutes per week.) Findings for both elementary and high school students were statistically significant. These weekly differences in time-use patterns very likely are cumulative over time. For example, the weekly difference of 2 hours 6 minutes for elementary students translates into 79 hours 48 minutes during a 38-week school year. The nearly 80-hour yearly difference in "engaged learning activity" may contribute to higher-scoring students, on average. As time goes by, high and low achievers may display more obvious differences in their amount of exposure to constructive out-of-school learning activities.
|
Salient Effect of Activity Quality on Student Achievement
Another revelation from the data is that student achievement on standardized tests of reading is correlated with the quality of students' active engagement in out-of-school, high-yield activities. Quality was operationalized by the parent's perception of how intently the student focuses on the activities, how enthusiastically the student performs the activities, and how frequently the student takes on leadership roles while doing the activities.
In the Los Angeles study, it was hypothesized that the quality of students' constructive learning activities is correlated with student achievement in reading. This hypothesis was tested with 31 fourth-grade Los Angeles public school students, their parents, and educators, who participated in a comprehensive ethnographic study on the effects of learning activities on achievement. Fourteen of the students were Latino (Mexican American), nine were Black, and eight were White. The parents were of the same ethnic group as their children.
The 31 students were classified as high achievers and lower achievers based on their scores on standardized tests and teacher ratings of students' classroom learning behavior (provided during face-to-face interviews with the 14 teachers). Twenty students were classified as high achievers based on their NCE scores above the 50th percentile on the total reading portion of the Comprehensive Test of Basic Skills and positive teacher ratings. Eleven students were classified as lower achievers based on their NCE scores at or below the 50th percentile on the total reading portion of the Comprehensive Test of Basic Skills and/or negative teacher ratings. The NCE scores on the total reading portion of the Comprehensive Test of Basic Skills were obtained from school records. Teacher assessments of each student's classroom skill level were obtained from teacher interviews.
Specifically, the quality of two types of out-of-school learning activities were examined: high-yield literacy activities and high-yield enrichment activities. For this analysis, the definition of high-yield literacy activities in out-of-school learning included reading, writing, and studying. High-yield enrichment activities in out-of-school learning included doing hobbies and playing games. Parents responded to ethnographic interview questions (audiotaped) about their child's behavior at home during each of the five activities. Quality was measured by ratings scores assigned to parents' oral responses pertaining to their child's level of enthusiasm, focus/effort, and leadership role behavior during each of the activities. Parents' responses to each of these three measures were rated as "often," "sometimes," or "seldom" (coded as 3, 2, and 1, respectively).
Although this sample was small and nonrandom, there were identifiable achievement test performance gaps related to race. Twenty students were classified as high achievers based on test scores: Eight high achievers were White, nine were Mexican American (seven English-language dominant and two Spanish-language dominant), and three were Black. Eleven students were classified as lower achievers: Six lower achievers were Black, and five were Mexican American (three English-language dominant and two Spanish-language dominant). None of the White students in this sample were classified as lower achievers.
The data show that high achievers generally were involved at a higher-quality level in the five constructive out-of-school activities more often than lower achievers. Students' quality of active engagement while doing high-yield activities was statistically significant for the set of activities.
Parental Beliefs and Expectations
The beliefs and attitudes of parents play a significant role in student success in becoming competent readers. In the Nashville study, parents of 459 students responded to questions about their expectations for their child's learning and their perception of whether they had been supported by their child's teacher. Specifically, they were asked the following questions:
- Please check the highest level of education you expect your child to eventually complete someday.
- How much help or encouragement have you received from your child's teacher?
Analysis of the data showed that parents' responses to these questions were significantly associated with students' reading achievement scores in the multivariate analysis that was conducted (see Exhibit 2). Clearly, students benefit when parents (1) set high standards for their child's performance in school, and (2) feel personally supported by partnerships they have formed with their child's teachers.
Teacher-Parent Communication
Parent beliefs are likely to be influenced by teacher-parent communication as well. In other words, parents may benefit from well-organized teacher-led communication actions, regardless of the parents' initial mind-set when their children start school. When teachers take actions to cultivate instructional partnerships with parents, those parents are more likely to support their children's learning at home; also, the students of these parents are more likely to be perceived by the teachers as positively involved in classroom learning activities. Evidence from the Nashville study supports this hypothesis. In the Nashville study, teachers were asked the following questions:
- What proportion of your students' parents did you provide with information or materials (not including homework) to help their children develop and refine skills needed in school for each subject area (reading, English/language arts, math)?
- How effective in motivating parents to help their children at home is the information you provided for the parents?
Parents were asked the following question:
- How well do you think you work with your child's teacher?
The data showed that students' scores were higher on the Tennessee Comprehensive Assessment Program standardized test of reading when teachers reported more communication with parents and when those parents perceived themselves to be engaged in a healthy partnership with the teacher. These factors accounted for about 7 percent of the variance in students' reading achievement scores (see Exhibit 2).
Role of Family Background and Neighborhood Safety
The multivariate analysis in Exhibit 2 shows that after variations in students' in-school and out-of-school experiences fully are accounted for, family background (ethnicity and socioeconomic status) alone contribute relatively little to variations in student achievement (9 percent). Parent perceptions of community and neighborhood safety similarly explained a relatively small amount of the difference in a student's test scores (10 percent) after taking into consideration activity-focused school factors and out-of-school factors. Essentially, family-background factors do not appear to be independently or primarily responsible for variations in student achievement levels. Rather, student achievement scores on standardized tests are most consistently and powerfully associated with the behaviors of students, teachers, and parents, as described in earlier sections of this article.
Summary
The data show that when an appropriately comprehensive range of in-school and out-of-school student and adult behaviors is taken into account, race and class do not strongly correlate with student achievement levels. Students', teachers', and parents' performance (or nonperformance) of the behaviors described in earlier sections of this article show the strongest correlations to student achievement. These data suggest that the achievement gap between students from different races and social classes largely may be most directly associated with variations in the time-use habits of students (in and out of school), and the involvement of parents, teachers, and adult mentors in students' activities.
Further research is needed with larger urban populations to confirm and expand the findings of these exploratory studies. Future studies should utilize multiple methods, including experimental designs with random samples; data gathering and analysis techniques that capture students' total array of learning habits in school, home, and community settings; and data gathering and analysis techniques that capture students' perceptions of the form and function of their out-of-school learning efforts during out-of-school activities. More studies should consider a rigorous ecological approach to student learning (i.e., examine the multiple settings where a specific cohort of randomly selected students regularly spend time) so that they may adequately capture the most significant determinants of students' school performance on standardized tests.
Policy Options
Collectively, these studies do not provide a complete or perfect set of correlates of student achievement. At best, the results from this work are suggestive of the deeper structural behavior patterns that are associated with variations in student achievement. Nevertheless, results from the ethnographic and quantitative work demonstrate that variations in student achievement on standardized tests (whether within-group or between groups) are closely associated with variations in what people do. This fundamental fact presents the prudent reader with clues about particular educational policies and practices in urban schools and community agencies that are likely to affect the achievement gap.
Schools that expect to close the student achievement gap in reading will need to create practices that first accomplish the following goals:
- Close the gap in the instructional habits and effectiveness of teachers.
- Close the gap in the out-of-school learning habits of students.
The data presented, although drawn from relatively small and nonrandom samples, show that students' academic success in reading (as measured by school norm-referenced test performance) frequently was seen when the following situations occurred:
- Students spent at least 3 hours a day with teachers doing structured (presumably well-organized and well-executed) learning activities or lessons.
- Students tended to spend anywhere from 8 to15 hours a week— depending on grade level—in high-yield, out-of-school learning activities (such as reading, writing, study or homework, and intellectually stimulating games and hobbies).
- Students displayed a high level of enthusiasm, focus/effort, and leadership role behavior during each of the activities.
- Students were not engaged in excessive amounts of unstructured leisure activity (e.g. hanging out or watching television), work or chore activities, or travel or commuting activities.
- Parents frequently communicated to their students that the parent required the student to fully participate in school learning activities, succeed on school tasks, and ultimately complete four years of college one day.
- The student's teacher reached out and contacted parents, built rapport with the parents, and invited parents into a working partnership. Then the teacher followed up with regular reports to parents about the student's classroom performance and overall academic progress, information about homework, and information on how to support the student's learning at home.
The focus of policymaking, in large part, must be on creating and maintaining programs and accountability systems that increase students' involvement in success-oriented lifestyles. These programs and accountability systems must require school staffs and parents to demonstrate that their students are engaged in the requisite types and amounts of constructive school-classroom learning activities and constructive out-of-school learning activities. If these programs are well-designed, they can add significantly to students' opportunity to learn. When students are not being engaged in a minimal threshold level of the constructive activities, schools and/or parents should be required to explain why. Then a plan for correcting the opportunity- to-learn threshold deficiencies should be created and implemented. Policymakers must provide appropriate training, resources, and incentives to generate the cultural shift that will be necessary to instigate these practices in most low-achieving school (and after-school) settings.
Reginald Clark, Ph.D., is president of Clark and Associates, an independent consulting firm based in Montclair, California. He currently is a lecturer in the Department of Child and Family Studies at California State University, Los Angeles. Dr. Clark has spent much of the last decade doing research on youth development and working as an applied sociologist, assisting schools and other education agencies in developing interventions for youths, parents, and teachers that hold promise for closing the achievement gap.
